Jul 07 14:26:37.511942 ip-10-0-141-129 systemd[1]: Starting Kubernetes Kubelet... Jul 07 14:26:37.892002 ip-10-0-141-129 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 14:26:37.892002 ip-10-0-141-129 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 14:26:37.892002 ip-10-0-141-129 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 14:26:37.892002 ip-10-0-141-129 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 14:26:37.892002 ip-10-0-141-129 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 14:26:37.893787 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.893704 2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 14:26:37.897883 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897869 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:26:37.897883 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897883 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897888 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897892 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897895 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897898 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897901 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897904 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897906 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897909 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897912 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897915 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897918 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897920 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897923 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897931 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897933 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897936 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897938 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897941 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897944 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:26:37.897939 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897946 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897949 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897953 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897957 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897960 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897963 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897966 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897969 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897972 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897975 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897978 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897981 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897984 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897987 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897990 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897992 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897994 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897997 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.897999 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898002 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:26:37.898407 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898004 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898006 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898009 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898011 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898014 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898016 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898019 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898021 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898024 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898027 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898029 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898031 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898034 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898037 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898040 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898043 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898045 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898049 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898052 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898055 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:26:37.898952 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898057 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898060 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898062 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898065 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898068 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898071 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898074 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898076 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898078 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898081 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898083 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898086 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898088 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898091 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898093 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898095 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898098 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898101 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898103 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898106 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:26:37.899430 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898109 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898111 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898114 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898118 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898121 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898125 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898129 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898132 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898135 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898138 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898140 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898143 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898146 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898149 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898151 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898154 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898156 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.898159 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899524 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:26:37.899918 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899531 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899535 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899538 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899541 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899546 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899550 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899553 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899556 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899559 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899561 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899565 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899568 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899571 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899574 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899577 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899579 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899582 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899585 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899588 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899591 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:26:37.900364 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899594 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899596 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899599 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899602 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899604 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899607 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899609 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899612 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899614 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899617 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899620 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899623 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899625 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899628 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899630 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899633 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899635 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899638 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899640 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899642 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:26:37.900874 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899645 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899647 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899650 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899654 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899656 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899659 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899661 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899677 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899679 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899682 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899685 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899688 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899691 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899694 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899696 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899699 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899701 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899704 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899706 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:26:37.901384 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899709 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899712 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899714 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899717 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899720 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899722 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899725 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899727 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899730 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899732 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899735 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899737 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899739 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899742 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899744 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899747 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899751 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899755 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899759 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:26:37.901858 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899761 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899764 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899766 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899769 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899771 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899775 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899778 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899780 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899784 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899787 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899790 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899793 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899795 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899798 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899800 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899803 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899805 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899808 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899810 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:26:37.902321 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.899813 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900206 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900212 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900215 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900218 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900221 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900224 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900227 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900230 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900232 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900236 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900239 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900241 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900244 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900247 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900250 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900252 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900255 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900258 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900260 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:26:37.902792 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900263 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900265 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900268 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900270 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900273 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900275 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900278 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900280 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900283 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900285 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900288 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900290 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900293 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900295 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900298 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900300 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900303 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900305 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900308 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900310 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:26:37.903281 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900313 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900315 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900320 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900325 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900329 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900331 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900334 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900337 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900339 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900342 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900344 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900347 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900349 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900352 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900355 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900357 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900360 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900362 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900365 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900367 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:26:37.903770 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900370 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900373 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900376 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900378 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900381 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900383 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900386 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900388 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900391 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900393 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900396 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900398 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900400 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900403 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900407 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900410 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900414 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900417 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900420 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:26:37.904253 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900424 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900427 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900429 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900432 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900434 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900437 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900440 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900443 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900446 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900449 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900452 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900454 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900457 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900459 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900462 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900465 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900467 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900470 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900472 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:26:37.904716 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900474 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.900477 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900543 2555 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900549 2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900556 2555 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900560 2555 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900564 2555 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900567 2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900571 2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900575 2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900578 2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900581 2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900585 2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900588 2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900591 2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900594 2555 flags.go:64] FLAG: --cgroup-root="" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900597 2555 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900600 2555 flags.go:64] FLAG: --client-ca-file="" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900602 2555 flags.go:64] FLAG: --cloud-provider="external" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900605 2555 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900609 2555 flags.go:64] FLAG: --cluster-domain="" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900612 2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900615 2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900618 2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 14:26:37.905170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900621 2555 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900624 2555 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900627 2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900630 2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900633 2555 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900636 2555 flags.go:64] FLAG: --contention-profiling="false" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900639 2555 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900642 2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900647 2555 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900651 2555 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900687 2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900690 2555 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900693 2555 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900696 2555 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900699 2555 flags.go:64] FLAG: --enable-server="true" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900701 2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900706 2555 flags.go:64] FLAG: --event-burst="100" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900709 2555 flags.go:64] FLAG: --event-qps="50" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900712 2555 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900715 2555 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900718 2555 flags.go:64] FLAG: --eviction-hard="" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900722 2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900724 2555 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900727 2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900730 2555 flags.go:64] FLAG: --eviction-soft="" Jul 07 14:26:37.905733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900733 2555 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900735 2555 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900738 2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900741 2555 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900743 2555 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900746 2555 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900749 2555 flags.go:64] FLAG: --feature-gates="" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900752 2555 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900755 2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900758 2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900761 2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900763 2555 flags.go:64] FLAG: --healthz-port="10248" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900766 2555 flags.go:64] FLAG: --help="false" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900769 2555 flags.go:64] FLAG: --hostname-override="ip-10-0-141-129.ec2.internal" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900772 2555 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900774 2555 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900777 2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900780 2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900783 2555 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900786 2555 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900789 2555 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900791 2555 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900794 2555 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 14:26:37.906324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900797 2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900800 2555 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900802 2555 flags.go:64] FLAG: --kube-reserved="" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900805 2555 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900808 2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900810 2555 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900813 2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900816 2555 flags.go:64] FLAG: --lock-file="" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900819 2555 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900821 2555 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900824 2555 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900829 2555 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900832 2555 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900834 2555 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900837 2555 flags.go:64] FLAG: --logging-format="text" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900839 2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900843 2555 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900845 2555 flags.go:64] FLAG: --manifest-url="" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900848 2555 flags.go:64] FLAG: --manifest-url-header="" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900852 2555 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900855 2555 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900859 2555 flags.go:64] FLAG: --max-pods="110" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900862 2555 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900865 2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900867 2555 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 14:26:37.906987 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900870 2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900873 2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900875 2555 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900878 2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900885 2555 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900888 2555 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900894 2555 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900897 2555 flags.go:64] FLAG: --pod-cidr="" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900900 2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900905 2555 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900908 2555 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900911 2555 flags.go:64] FLAG: --pods-per-core="0" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900913 2555 flags.go:64] FLAG: --port="10250" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900916 2555 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900920 2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0497018b9225ca681" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900923 2555 flags.go:64] FLAG: --qos-reserved="" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900927 2555 flags.go:64] FLAG: --read-only-port="10255" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900929 2555 flags.go:64] FLAG: --register-node="true" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900932 2555 flags.go:64] FLAG: --register-with-taints="" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900936 2555 flags.go:64] FLAG: --registry-burst="10" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900939 2555 flags.go:64] FLAG: --registry-qps="5" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900941 2555 flags.go:64] FLAG: --reserved-cpus="" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900945 2555 flags.go:64] FLAG: --reserved-memory="" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900949 2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900952 2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 14:26:37.907740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900955 2555 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900958 2555 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900960 2555 flags.go:64] FLAG: --runonce="false" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900963 2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900966 2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900969 2555 flags.go:64] FLAG: --seccomp-default="false" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900972 2555 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900974 2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900977 2555 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900980 2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900983 2555 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900986 2555 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900989 2555 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900992 2555 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900995 2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.900999 2555 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901001 2555 flags.go:64] FLAG: --system-cgroups="" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901004 2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901009 2555 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901012 2555 flags.go:64] FLAG: --tls-cert-file="" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901015 2555 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901018 2555 flags.go:64] FLAG: --tls-min-version="" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901021 2555 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901024 2555 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901027 2555 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 14:26:37.908354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901030 2555 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901032 2555 flags.go:64] FLAG: --v="2" Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901036 2555 flags.go:64] FLAG: --version="false" Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901040 2555 flags.go:64] FLAG: --vmodule="" Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901045 2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901049 2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901160 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901164 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901168 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901171 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901175 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901179 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901182 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901184 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901187 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901189 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901192 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901195 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901197 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901199 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901202 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:26:37.908966 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901206 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901209 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901211 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901214 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901216 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901219 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901221 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901223 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901226 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901229 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901231 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901234 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901236 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901239 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901241 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901245 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901248 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901250 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901252 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901255 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:26:37.909474 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901258 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901260 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901263 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901265 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901269 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901272 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901275 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901278 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901280 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901283 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901285 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901288 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901291 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901294 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901296 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901299 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901302 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901304 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901307 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901309 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:26:37.909977 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901312 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901314 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901317 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901320 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901322 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901324 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901327 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901331 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901334 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901336 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901339 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901341 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901344 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901346 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901349 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901351 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901353 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901356 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901359 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901361 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:26:37.910505 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901364 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901366 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901369 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901371 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901376 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901379 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901382 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901386 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901388 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901391 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901394 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901397 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901400 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901402 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901404 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901407 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901410 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901412 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901415 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:26:37.911013 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901419 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:26:37.911478 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901421 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:26:37.911478 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901424 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:26:37.911478 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901426 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:26:37.911478 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.901429 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:26:37.911478 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.901948 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 14:26:37.911478 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.908531 2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 14:26:37.911478 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.908546 2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 14:26:37.911478 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.908564 2555 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 14:26:37.911478 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.908574 2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 14:26:37.911478 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908629 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:26:37.911478 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908634 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:26:37.911478 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908638 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:26:37.911478 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908641 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:26:37.911478 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908643 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:26:37.911478 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908646 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908649 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908651 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908654 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908657 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908659 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908662 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908683 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908686 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908689 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908692 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908695 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908698 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908700 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908703 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908707 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908711 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908714 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908717 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:26:37.911880 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908720 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908724 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908727 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908730 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908732 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908736 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908739 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908742 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908745 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908747 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908750 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908752 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908755 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908758 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908760 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908763 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908765 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908768 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908770 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908773 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:26:37.912330 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908775 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908778 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908780 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908783 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908785 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908788 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908790 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908793 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908795 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908798 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908800 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908803 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908806 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908809 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908812 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908814 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908817 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908833 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908838 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908842 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:26:37.912833 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908845 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908848 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908851 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908853 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908856 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908859 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908862 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908864 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908867 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908869 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908872 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908874 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908877 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908879 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908882 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908884 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908887 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908890 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908892 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908895 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:26:37.913299 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908897 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:26:37.913799 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908900 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:26:37.913799 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908902 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:26:37.913799 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908904 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:26:37.913799 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908908 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:26:37.913799 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908911 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:26:37.913799 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908914 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:26:37.913799 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908917 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:26:37.913799 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908920 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:26:37.913799 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908923 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:26:37.913799 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908926 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:26:37.913799 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908929 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:26:37.913799 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908932 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:26:37.913799 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908935 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:26:37.913799 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.908937 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:26:37.914135 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.908942 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 14:26:37.914135 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909053 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 14:26:37.914135 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909057 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 14:26:37.914135 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909060 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 14:26:37.914135 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909063 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 14:26:37.914135 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909066 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 14:26:37.914135 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909069 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 14:26:37.914135 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909072 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 14:26:37.914135 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909074 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 14:26:37.914135 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909077 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 14:26:37.914135 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909079 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 14:26:37.914135 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909081 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 14:26:37.914135 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909084 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 14:26:37.914135 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909086 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 14:26:37.914135 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909088 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909091 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909093 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909096 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909098 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909101 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909103 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909106 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909109 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909113 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909116 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909119 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909121 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909124 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909126 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909129 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909131 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909134 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909136 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909139 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 14:26:37.914615 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909141 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909143 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909146 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909148 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909150 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909153 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909155 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909157 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909160 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909162 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909165 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909167 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909169 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909172 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909174 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909177 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909179 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909182 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909184 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909187 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 14:26:37.915443 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909189 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909192 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909195 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909197 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909199 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909203 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909206 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909209 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909212 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909215 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909218 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909221 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909224 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909226 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909228 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909231 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909233 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909236 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909238 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 14:26:37.916060 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909240 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909243 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909245 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909248 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909250 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909253 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909255 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909257 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909260 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909262 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909265 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909267 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909270 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909272 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909275 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909277 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909279 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909282 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909285 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 14:26:37.916698 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909287 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 14:26:37.917156 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909289 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 14:26:37.917156 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909292 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 14:26:37.917156 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909295 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 14:26:37.917156 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909297 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 14:26:37.917156 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909299 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 14:26:37.917156 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909302 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 14:26:37.917156 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:37.909305 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 14:26:37.917156 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.909310 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 14:26:37.917156 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.909409 2555 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 14:26:37.917156 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.911408 2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 14:26:37.917156 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.912731 2555 server.go:1019] "Starting client certificate rotation" Jul 07 14:26:37.917156 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.912833 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 14:26:37.917156 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.912876 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 14:26:37.917659 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.917645 2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 14:26:37.918807 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.918791 2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 14:26:37.930837 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.930822 2555 log.go:25] "Validated CRI v1 runtime API" Jul 07 14:26:37.932746 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.932732 2555 log.go:25] "Validated CRI v1 image API" Jul 07 14:26:37.933463 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.933446 2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 14:26:37.936441 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.936423 2555 fs.go:135] Filesystem UUIDs: map[22e9bf4b-0d65-4545-a698-ab86e25ee6a4:/dev/nvme0n1p3 5b686ae6-e899-4a03-9054-e115f5c33934:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 07 14:26:37.936484 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.936442 2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 14:26:37.940725 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.940705 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 14:26:37.944414 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.944311 2555 manager.go:216] Machine: {Timestamp:2026-07-07 14:26:37.941242461 +0000 UTC m=+0.320695952 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103634 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec273acafdd6c376418d4c1424d769f2 SystemUUID:ec273aca-fdd6-c376-418d-4c1424d769f2 BootID:d15be04e-f2b9-4b0c-9d08-92323d5f06e6 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3c:5b:02:20:4b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3c:5b:02:20:4b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:8a:8a:e3:86:82 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 14:26:37.944414 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.944406 2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 14:26:37.944533 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.944482 2555 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 14:26:37.947567 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.947539 2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 14:26:37.947763 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.947569 2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-129.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 14:26:37.948908 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.948898 2555 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 14:26:37.948939 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.948911 2555 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 14:26:37.948939 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.948919 2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 14:26:37.948939 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.948929 2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 14:26:37.950255 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.950246 2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 14:26:37.952912 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.952901 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 07 14:26:37.953034 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.953025 2555 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 14:26:37.953073 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.953055 2555 kubelet.go:486] "Attempting to sync node with API server" Jul 07 14:26:37.953105 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.953074 2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 14:26:37.953105 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.953089 2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 14:26:37.953105 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.953098 2555 kubelet.go:398] "Adding apiserver pod source" Jul 07 14:26:37.953232 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.953109 2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 14:26:37.953771 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.953746 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 14:26:37.953876 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.953788 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 14:26:37.957609 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.957590 2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 14:26:37.959423 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.959409 2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 14:26:37.959475 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.959449 2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 14:26:37.960633 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.960622 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 14:26:37.960686 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.960638 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 14:26:37.960686 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.960644 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 14:26:37.960686 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.960649 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 14:26:37.960686 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.960658 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 14:26:37.960686 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.960663 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 14:26:37.960824 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.960691 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 14:26:37.960824 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.960699 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 14:26:37.960824 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.960705 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 14:26:37.960824 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.960710 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 14:26:37.960824 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.960719 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 14:26:37.960824 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.960724 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 14:26:37.961379 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.961368 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 14:26:37.961410 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.961380 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 14:26:37.962770 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.962729 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wzt24" Jul 07 14:26:37.965006 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.964991 2555 server.go:1268] "Started kubelet" Jul 07 14:26:37.965457 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.965142 2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 14:26:37.965457 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.965209 2555 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 14:26:37.965457 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.965221 2555 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-129.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 14:26:37.965457 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:37.965281 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-129.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 14:26:37.965457 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.965310 2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 14:26:37.965751 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:37.965475 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 14:26:37.965947 ip-10-0-141-129 systemd[1]: Started Kubernetes Kubelet. Jul 07 14:26:37.966496 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.966428 2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 14:26:37.967430 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.967417 2555 server.go:310] "Adding debug handlers to kubelet server" Jul 07 14:26:37.970578 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.970561 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wzt24" Jul 07 14:26:37.971047 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:37.970084 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-129.ec2.internal.18c00822caea05ff default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-129.ec2.internal,UID:ip-10-0-141-129.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-129.ec2.internal,},FirstTimestamp:2026-07-07 14:26:37.964961279 +0000 UTC m=+0.344414770,LastTimestamp:2026-07-07 14:26:37.964961279 +0000 UTC m=+0.344414770,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-129.ec2.internal,}" Jul 07 14:26:37.972318 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:37.972293 2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 14:26:37.974760 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.974743 2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 14:26:37.974843 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.974749 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 14:26:37.975252 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:37.974929 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-129.ec2.internal\" not found" Jul 07 14:26:37.975450 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.975433 2555 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 14:26:37.975543 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.975453 2555 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 14:26:37.975543 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.975433 2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 14:26:37.975686 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.975657 2555 reconstruct.go:97] "Volume reconstruction finished" Jul 07 14:26:37.975744 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.975686 2555 reconciler.go:29] "Reconciler: start to sync state" Jul 07 14:26:37.976047 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.976013 2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 14:26:37.976047 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.976030 2555 factory.go:55] Registering systemd factory Jul 07 14:26:37.976047 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.976039 2555 factory.go:223] Registration of the systemd container factory successfully Jul 07 14:26:37.976265 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.976254 2555 factory.go:153] Registering CRI-O factory Jul 07 14:26:37.976300 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.976270 2555 factory.go:223] Registration of the crio container factory successfully Jul 07 14:26:37.976300 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.976288 2555 factory.go:103] Registering Raw factory Jul 07 14:26:37.976356 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.976311 2555 manager.go:1196] Started watching for new ooms in manager Jul 07 14:26:37.976629 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.976613 2555 manager.go:349] Starting recovery of all containers Jul 07 14:26:37.977512 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:37.977473 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 14:26:37.977632 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:37.977593 2555 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-129.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 14:26:37.988548 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.988442 2555 manager.go:354] Recovery completed Jul 07 14:26:37.993047 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:37.993031 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:26:38.001640 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.001625 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:26:38.001749 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.001653 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:26:38.001749 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.001680 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:26:38.002222 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.002206 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:26:38.002298 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.002207 2555 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 14:26:38.002298 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.002237 2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 14:26:38.002298 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.002252 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 07 14:26:38.002905 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.002892 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:26:38.002976 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.002918 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:26:38.002976 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.002930 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:26:38.003325 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.003313 2555 policy_none.go:49] "None policy: Start" Jul 07 14:26:38.003355 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.003331 2555 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 14:26:38.003355 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.003342 2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 14:26:38.004373 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.004363 2555 policy_none.go:47] "Start" Jul 07 14:26:38.041183 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.041168 2555 manager.go:337] "Starting Device Plugin manager" Jul 07 14:26:38.041268 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:38.041197 2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 14:26:38.041268 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.041207 2555 server.go:87] "Starting device plugin registration server" Jul 07 14:26:38.041453 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.041441 2555 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 14:26:38.041491 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.041455 2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 14:26:38.041564 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.041546 2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 14:26:38.041643 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.041630 2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 14:26:38.041705 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.041645 2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 14:26:38.042131 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:38.042111 2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 14:26:38.042215 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:38.042165 2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-129.ec2.internal\" not found" Jul 07 14:26:38.140426 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.140395 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 14:26:38.141625 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.141611 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:26:38.141858 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.141840 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 14:26:38.141917 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.141876 2555 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 14:26:38.141917 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.141898 2555 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 14:26:38.142044 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:38.141931 2555 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 14:26:38.142756 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.142729 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:26:38.142896 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.142761 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:26:38.142896 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.142773 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:26:38.142896 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.142803 2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.144351 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.144329 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:26:38.150993 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.150973 2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.151083 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:38.150994 2555 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-129.ec2.internal\": node \"ip-10-0-141-129.ec2.internal\" not found" Jul 07 14:26:38.197003 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:38.196982 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-129.ec2.internal\" not found" Jul 07 14:26:38.242051 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.242025 2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-129.ec2.internal"] Jul 07 14:26:38.242133 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.242093 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:26:38.243054 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.243040 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:26:38.243148 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.243069 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:26:38.243148 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.243079 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:26:38.244459 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.244446 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:26:38.244572 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.244558 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.244617 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.244586 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:26:38.245117 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.245103 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:26:38.245117 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.245124 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:26:38.245242 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.245137 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:26:38.245242 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.245187 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:26:38.245242 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.245204 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:26:38.245242 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.245213 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:26:38.246436 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.246422 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.246518 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.246446 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 14:26:38.247098 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.247082 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasSufficientMemory" Jul 07 14:26:38.247161 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.247111 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 14:26:38.247161 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.247128 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeHasSufficientPID" Jul 07 14:26:38.272280 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:38.272266 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-129.ec2.internal\" not found" node="ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.276779 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.276762 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/831aa4a8e60ac1bafce8327f1e975c1a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal\" (UID: \"831aa4a8e60ac1bafce8327f1e975c1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.276860 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.276787 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/831aa4a8e60ac1bafce8327f1e975c1a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal\" (UID: \"831aa4a8e60ac1bafce8327f1e975c1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.276860 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.276803 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4a862d27acdd6adcd1a30a65484081bd-config\") pod \"kube-apiserver-proxy-ip-10-0-141-129.ec2.internal\" (UID: \"4a862d27acdd6adcd1a30a65484081bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.276860 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:38.276856 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-129.ec2.internal\" not found" node="ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.297675 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:38.297655 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-129.ec2.internal\" not found" Jul 07 14:26:38.377566 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.377539 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4a862d27acdd6adcd1a30a65484081bd-config\") pod \"kube-apiserver-proxy-ip-10-0-141-129.ec2.internal\" (UID: \"4a862d27acdd6adcd1a30a65484081bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.377566 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.377569 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/831aa4a8e60ac1bafce8327f1e975c1a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal\" (UID: \"831aa4a8e60ac1bafce8327f1e975c1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.377758 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.377601 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/831aa4a8e60ac1bafce8327f1e975c1a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal\" (UID: \"831aa4a8e60ac1bafce8327f1e975c1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.377758 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.377628 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4a862d27acdd6adcd1a30a65484081bd-config\") pod \"kube-apiserver-proxy-ip-10-0-141-129.ec2.internal\" (UID: \"4a862d27acdd6adcd1a30a65484081bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.377758 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.377648 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/831aa4a8e60ac1bafce8327f1e975c1a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal\" (UID: \"831aa4a8e60ac1bafce8327f1e975c1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.377758 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.377630 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/831aa4a8e60ac1bafce8327f1e975c1a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal\" (UID: \"831aa4a8e60ac1bafce8327f1e975c1a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.398705 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:38.398638 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-129.ec2.internal\" not found" Jul 07 14:26:38.499475 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:38.499439 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-129.ec2.internal\" not found" Jul 07 14:26:38.574654 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.574627 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.579328 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:38.579312 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-129.ec2.internal" Jul 07 14:26:38.600194 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:38.600165 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-129.ec2.internal\" not found" Jul 07 14:26:38.700732 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:38.700695 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-129.ec2.internal\" not found" Jul 07 14:26:38.801192 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:38.801151 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-129.ec2.internal\" not found" Jul 07 14:26:39.898900 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.898881 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:26:39.915954 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.915932 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:26:39.933054 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:39.933028 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-129.ec2.internal\" not found" Jul 07 14:26:39.944195 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.944172 2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 14:26:39.944315 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.944299 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 14:26:39.944359 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.944338 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 14:26:39.944404 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.944341 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 14:26:39.957810 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.957785 2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:26:39.985298 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.985246 2555 apiserver.go:52] "Watching apiserver" Jul 07 14:26:39.993984 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.993964 2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 14:26:39.994282 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.994263 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-8c9rl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh","openshift-image-registry/node-ca-mbx8j","openshift-multus/network-metrics-daemon-98k9b","openshift-network-diagnostics/network-check-target-bt4tq","kube-system/konnectivity-agent-sc8gn","openshift-cluster-node-tuning-operator/tuned-h976w","openshift-dns/node-resolver-dz5b9","openshift-multus/multus-additional-cni-plugins-b9xjq","openshift-multus/multus-h7jl6","openshift-network-operator/iptables-alerter-8m2bd","openshift-ovn-kubernetes/ovnkube-node-9ml5j"] Jul 07 14:26:39.995710 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.995695 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:39.995801 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:39.995783 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:26:39.996827 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.996811 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:39.998401 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.998378 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mbx8j" Jul 07 14:26:39.998787 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.998764 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 14:26:39.999139 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.999123 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vfm84\"" Jul 07 14:26:39.999253 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.999129 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 14:26:39.999253 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.999208 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 14:26:39.999478 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:39.999464 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:39.999558 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:39.999537 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:26:40.000046 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.000031 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 14:26:40.000233 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.000220 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 14:26:40.000316 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.000286 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 14:26:40.000411 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.000374 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4t5rd\"" Jul 07 14:26:40.000780 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.000763 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:40.000857 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.000837 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:26:40.001940 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.001925 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sc8gn" Jul 07 14:26:40.003170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.003151 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.003605 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.003589 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9gqcj\"" Jul 07 14:26:40.003680 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.003612 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 14:26:40.003680 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.003612 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 14:26:40.004374 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.004359 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dz5b9" Jul 07 14:26:40.004745 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.004732 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xx5hm\"" Jul 07 14:26:40.004916 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.004901 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:26:40.005657 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.005642 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 14:26:40.005745 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.005664 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.006038 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.006021 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sdw92\"" Jul 07 14:26:40.006211 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.006198 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 14:26:40.006304 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.006288 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal" Jul 07 14:26:40.006398 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.006392 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 14:26:40.006448 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.006396 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 14:26:40.006974 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.006955 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.007453 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.007428 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 14:26:40.007685 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.007639 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 14:21:38 +0000 UTC" deadline="2028-02-19 13:47:13.954355689 +0000 UTC" Jul 07 14:26:40.007756 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.007688 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14207h20m33.946671919s" Jul 07 14:26:40.007756 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.007696 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 14:26:40.007756 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.007704 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 14:26:40.009031 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.008260 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 14:26:40.009031 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.008315 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gfjqt\"" Jul 07 14:26:40.009031 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.008494 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 14:26:40.009031 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.008498 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8m2bd" Jul 07 14:26:40.009031 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.008796 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5q99j\"" Jul 07 14:26:40.010176 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.009549 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 14:26:40.011116 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.010969 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c4tjf\"" Jul 07 14:26:40.011390 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.011372 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:26:40.011468 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.011410 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 14:26:40.011468 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.011431 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 14:26:40.012145 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012127 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.012462 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012443 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-sys-fs\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.012553 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012475 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-var-lib-kubelet\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.012553 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012502 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f701165c-9e6c-4815-9bbc-f97ad6de524e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.012553 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012526 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-os-release\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.012553 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012550 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-cni-binary-copy\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.012689 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012574 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-multus-socket-dir-parent\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.012689 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012613 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks79h\" (UniqueName: \"kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h\") pod \"network-check-target-bt4tq\" (UID: \"c7ab7625-0322-4b19-8f4b-7f090aebbe76\") " pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:40.012689 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012663 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f701165c-9e6c-4815-9bbc-f97ad6de524e-system-cni-dir\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.012689 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012685 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-system-cni-dir\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.012848 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012710 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-hostroot\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.012848 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012732 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mgbj\" (UniqueName: \"kubernetes.io/projected/fdcc1ba3-fd82-4fcb-be4b-92340a548edd-kube-api-access-5mgbj\") pod \"node-resolver-dz5b9\" (UID: \"fdcc1ba3-fd82-4fcb-be4b-92340a548edd\") " pod="openshift-dns/node-resolver-dz5b9" Jul 07 14:26:40.012848 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012748 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t5c5\" (UniqueName: \"kubernetes.io/projected/6b50a281-2f0c-4b83-963b-8ebecce1eb14-kube-api-access-5t5c5\") pod \"network-metrics-daemon-98k9b\" (UID: \"6b50a281-2f0c-4b83-963b-8ebecce1eb14\") " pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:40.012848 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012763 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f701165c-9e6c-4815-9bbc-f97ad6de524e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.012848 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012836 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.013014 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012881 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c055b3b3-1fac-45d3-82c1-a3e18bb068f8-konnectivity-ca\") pod \"konnectivity-agent-sc8gn\" (UID: \"c055b3b3-1fac-45d3-82c1-a3e18bb068f8\") " pod="kube-system/konnectivity-agent-sc8gn" Jul 07 14:26:40.013014 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012919 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-var-lib-cni-multus\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.013014 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012946 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-lib-modules\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.013014 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012969 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f701165c-9e6c-4815-9bbc-f97ad6de524e-os-release\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.013014 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.012990 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-var-lib-kubelet\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.013182 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013020 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f701165c-9e6c-4815-9bbc-f97ad6de524e-cnibin\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.013182 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013056 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-run-k8s-cni-cncf-io\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.013182 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013146 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hqrq\" (UniqueName: \"kubernetes.io/projected/be71c614-9b1c-4c09-9e28-8e9bb9021149-kube-api-access-7hqrq\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.013182 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013169 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-modprobe-d\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.013295 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013185 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-sysconfig\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.013295 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013199 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f701165c-9e6c-4815-9bbc-f97ad6de524e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.013295 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013230 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5p78\" (UniqueName: \"kubernetes.io/projected/f701165c-9e6c-4815-9bbc-f97ad6de524e-kube-api-access-n5p78\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.013295 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013245 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-run-multus-certs\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.013295 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013265 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-dbus\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:40.013295 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013279 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-sysctl-conf\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.013295 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013294 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-host\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.013573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013311 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:40.013573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013325 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-socket-dir\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.013573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013338 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-registration-dir\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.013573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013366 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-etc-kubernetes\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.013573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013409 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-sys\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.013573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013447 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-tuned\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.013573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013474 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-cnibin\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.013573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013509 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fdcc1ba3-fd82-4fcb-be4b-92340a548edd-hosts-file\") pod \"node-resolver-dz5b9\" (UID: \"fdcc1ba3-fd82-4fcb-be4b-92340a548edd\") " pod="openshift-dns/node-resolver-dz5b9" Jul 07 14:26:40.013573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013538 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67a6cbcf-d6c3-4b71-bed3-93e4935db1ec-host-slash\") pod \"iptables-alerter-8m2bd\" (UID: \"67a6cbcf-d6c3-4b71-bed3-93e4935db1ec\") " pod="openshift-network-operator/iptables-alerter-8m2bd" Jul 07 14:26:40.013573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013570 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-device-dir\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013593 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-etc-selinux\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013621 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft6f9\" (UniqueName: \"kubernetes.io/projected/f029d40b-288d-4026-97a5-119ff263ed93-kube-api-access-ft6f9\") pod \"node-ca-mbx8j\" (UID: \"f029d40b-288d-4026-97a5-119ff263ed93\") " pod="openshift-image-registry/node-ca-mbx8j" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013646 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t4z46\"" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013649 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldxzm\" (UniqueName: \"kubernetes.io/projected/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-kube-api-access-ldxzm\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013683 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-multus-conf-dir\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013721 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-systemd\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013745 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f701165c-9e6c-4815-9bbc-f97ad6de524e-cni-binary-copy\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013769 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-multus-cni-dir\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013794 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bspsb\" (UniqueName: \"kubernetes.io/projected/67a6cbcf-d6c3-4b71-bed3-93e4935db1ec-kube-api-access-bspsb\") pod \"iptables-alerter-8m2bd\" (UID: \"67a6cbcf-d6c3-4b71-bed3-93e4935db1ec\") " pod="openshift-network-operator/iptables-alerter-8m2bd" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013817 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-sysctl-d\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013842 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-run-netns\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013867 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f029d40b-288d-4026-97a5-119ff263ed93-host\") pod \"node-ca-mbx8j\" (UID: \"f029d40b-288d-4026-97a5-119ff263ed93\") " pod="openshift-image-registry/node-ca-mbx8j" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013890 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f029d40b-288d-4026-97a5-119ff263ed93-serviceca\") pod \"node-ca-mbx8j\" (UID: \"f029d40b-288d-4026-97a5-119ff263ed93\") " pod="openshift-image-registry/node-ca-mbx8j" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013927 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs\") pod \"network-metrics-daemon-98k9b\" (UID: \"6b50a281-2f0c-4b83-963b-8ebecce1eb14\") " pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013951 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-kubernetes\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013974 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-run\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.014029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.013999 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdcc1ba3-fd82-4fcb-be4b-92340a548edd-tmp-dir\") pod \"node-resolver-dz5b9\" (UID: \"fdcc1ba3-fd82-4fcb-be4b-92340a548edd\") " pod="openshift-dns/node-resolver-dz5b9" Jul 07 14:26:40.014772 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.014060 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 14:26:40.014772 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.014090 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-kubelet-config\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:40.014772 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.014097 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 14:26:40.014772 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.014123 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-tmp\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.014772 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.014134 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 14:26:40.014772 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.014149 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-var-lib-cni-bin\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.014772 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.014198 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4rv2\" (UniqueName: \"kubernetes.io/projected/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-kube-api-access-z4rv2\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.014772 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.014116 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 14:26:40.014772 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.014267 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c055b3b3-1fac-45d3-82c1-a3e18bb068f8-agent-certs\") pod \"konnectivity-agent-sc8gn\" (UID: \"c055b3b3-1fac-45d3-82c1-a3e18bb068f8\") " pod="kube-system/konnectivity-agent-sc8gn" Jul 07 14:26:40.014772 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.014310 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/67a6cbcf-d6c3-4b71-bed3-93e4935db1ec-iptables-alerter-script\") pod \"iptables-alerter-8m2bd\" (UID: \"67a6cbcf-d6c3-4b71-bed3-93e4935db1ec\") " pod="openshift-network-operator/iptables-alerter-8m2bd" Jul 07 14:26:40.014772 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.014332 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 14:26:40.014772 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.014336 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-multus-daemon-config\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.016277 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.016258 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 14:26:40.019542 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.019527 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 14:26:40.019622 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.019596 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-129.ec2.internal" Jul 07 14:26:40.019672 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.019658 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal"] Jul 07 14:26:40.021230 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.021216 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 14:26:40.027452 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.027435 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 14:26:40.027452 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.027446 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-129.ec2.internal"] Jul 07 14:26:40.052752 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.052736 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s775l" Jul 07 14:26:40.062453 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.062436 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s775l" Jul 07 14:26:40.108440 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.108422 2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 14:26:40.114568 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114544 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-run-systemd\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.114687 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114576 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-etc-openvswitch\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.114687 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114601 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/86594764-14f9-43fb-a595-518d0c910187-ovnkube-config\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.114687 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114630 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:40.114687 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114657 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-socket-dir\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.114687 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114680 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-registration-dir\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.114960 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114724 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-etc-kubernetes\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.114960 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114751 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-log-socket\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.114960 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114802 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-socket-dir\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.114960 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114813 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-cni-bin\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.114960 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114807 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-registration-dir\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.114960 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114842 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-etc-kubernetes\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.114960 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114847 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-sys\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.114960 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114888 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-sys\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.114960 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.114891 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:40.114960 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114896 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-tuned\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.114960 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114939 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-cnibin\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.114960 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114960 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fdcc1ba3-fd82-4fcb-be4b-92340a548edd-hosts-file\") pod \"node-resolver-dz5b9\" (UID: \"fdcc1ba3-fd82-4fcb-be4b-92340a548edd\") " pod="openshift-dns/node-resolver-dz5b9" Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.114998 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67a6cbcf-d6c3-4b71-bed3-93e4935db1ec-host-slash\") pod \"iptables-alerter-8m2bd\" (UID: \"67a6cbcf-d6c3-4b71-bed3-93e4935db1ec\") " pod="openshift-network-operator/iptables-alerter-8m2bd" Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115019 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-cnibin\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115042 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-run-ovn-kubernetes\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.115094 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret podName:cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:40.615045028 +0000 UTC m=+1.963098572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret") pod "global-pull-secret-syncer-8c9rl" (UID: "cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115123 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/86594764-14f9-43fb-a595-518d0c910187-ovn-node-metrics-cert\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115158 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q7sd\" (UniqueName: \"kubernetes.io/projected/86594764-14f9-43fb-a595-518d0c910187-kube-api-access-7q7sd\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115192 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-device-dir\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115213 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fdcc1ba3-fd82-4fcb-be4b-92340a548edd-hosts-file\") pod \"node-resolver-dz5b9\" (UID: \"fdcc1ba3-fd82-4fcb-be4b-92340a548edd\") " pod="openshift-dns/node-resolver-dz5b9" Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115235 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/67a6cbcf-d6c3-4b71-bed3-93e4935db1ec-host-slash\") pod \"iptables-alerter-8m2bd\" (UID: \"67a6cbcf-d6c3-4b71-bed3-93e4935db1ec\") " pod="openshift-network-operator/iptables-alerter-8m2bd" Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115248 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-device-dir\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115266 2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115292 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-etc-selinux\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115318 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft6f9\" (UniqueName: \"kubernetes.io/projected/f029d40b-288d-4026-97a5-119ff263ed93-kube-api-access-ft6f9\") pod \"node-ca-mbx8j\" (UID: \"f029d40b-288d-4026-97a5-119ff263ed93\") " pod="openshift-image-registry/node-ca-mbx8j" Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115342 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldxzm\" (UniqueName: \"kubernetes.io/projected/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-kube-api-access-ldxzm\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115366 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-multus-conf-dir\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.115546 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115400 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-etc-selinux\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115391 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-slash\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115486 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-systemd\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115518 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f701165c-9e6c-4815-9bbc-f97ad6de524e-cni-binary-copy\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115575 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-multus-cni-dir\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115606 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bspsb\" (UniqueName: \"kubernetes.io/projected/67a6cbcf-d6c3-4b71-bed3-93e4935db1ec-kube-api-access-bspsb\") pod \"iptables-alerter-8m2bd\" (UID: \"67a6cbcf-d6c3-4b71-bed3-93e4935db1ec\") " pod="openshift-network-operator/iptables-alerter-8m2bd" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115668 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-run-openvswitch\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115716 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-sysctl-d\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115771 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-run-netns\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115799 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-run-netns\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115949 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-var-lib-openvswitch\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115953 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-multus-conf-dir\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115973 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f029d40b-288d-4026-97a5-119ff263ed93-host\") pod \"node-ca-mbx8j\" (UID: \"f029d40b-288d-4026-97a5-119ff263ed93\") " pod="openshift-image-registry/node-ca-mbx8j" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.115996 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f029d40b-288d-4026-97a5-119ff263ed93-serviceca\") pod \"node-ca-mbx8j\" (UID: \"f029d40b-288d-4026-97a5-119ff263ed93\") " pod="openshift-image-registry/node-ca-mbx8j" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116022 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-systemd\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116034 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs\") pod \"network-metrics-daemon-98k9b\" (UID: \"6b50a281-2f0c-4b83-963b-8ebecce1eb14\") " pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116088 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-kubernetes\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.116378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116090 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-run-netns\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116140 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-multus-cni-dir\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116167 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-run\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.116185 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116221 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-run\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116225 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f029d40b-288d-4026-97a5-119ff263ed93-host\") pod \"node-ca-mbx8j\" (UID: \"f029d40b-288d-4026-97a5-119ff263ed93\") " pod="openshift-image-registry/node-ca-mbx8j" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.116283 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs podName:6b50a281-2f0c-4b83-963b-8ebecce1eb14 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:40.616264716 +0000 UTC m=+1.964318247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs") pod "network-metrics-daemon-98k9b" (UID: "6b50a281-2f0c-4b83-963b-8ebecce1eb14") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116370 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-sysctl-d\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116383 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdcc1ba3-fd82-4fcb-be4b-92340a548edd-tmp-dir\") pod \"node-resolver-dz5b9\" (UID: \"fdcc1ba3-fd82-4fcb-be4b-92340a548edd\") " pod="openshift-dns/node-resolver-dz5b9" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116406 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-kubelet-config\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116427 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-tmp\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116438 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-kubernetes\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116476 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-var-lib-cni-bin\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116525 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4rv2\" (UniqueName: \"kubernetes.io/projected/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-kube-api-access-z4rv2\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116550 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c055b3b3-1fac-45d3-82c1-a3e18bb068f8-agent-certs\") pod \"konnectivity-agent-sc8gn\" (UID: \"c055b3b3-1fac-45d3-82c1-a3e18bb068f8\") " pod="kube-system/konnectivity-agent-sc8gn" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116576 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/67a6cbcf-d6c3-4b71-bed3-93e4935db1ec-iptables-alerter-script\") pod \"iptables-alerter-8m2bd\" (UID: \"67a6cbcf-d6c3-4b71-bed3-93e4935db1ec\") " pod="openshift-network-operator/iptables-alerter-8m2bd" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116659 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f029d40b-288d-4026-97a5-119ff263ed93-serviceca\") pod \"node-ca-mbx8j\" (UID: \"f029d40b-288d-4026-97a5-119ff263ed93\") " pod="openshift-image-registry/node-ca-mbx8j" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116688 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdcc1ba3-fd82-4fcb-be4b-92340a548edd-tmp-dir\") pod \"node-resolver-dz5b9\" (UID: \"fdcc1ba3-fd82-4fcb-be4b-92340a548edd\") " pod="openshift-dns/node-resolver-dz5b9" Jul 07 14:26:40.117213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116726 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-var-lib-cni-bin\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116742 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-cni-netd\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116769 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-multus-daemon-config\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116806 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-sys-fs\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116862 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-var-lib-kubelet\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116890 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f701165c-9e6c-4815-9bbc-f97ad6de524e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116912 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-os-release\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116934 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-cni-binary-copy\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.116960 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-multus-socket-dir-parent\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117016 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-run-ovn\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117147 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks79h\" (UniqueName: \"kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h\") pod \"network-check-target-bt4tq\" (UID: \"c7ab7625-0322-4b19-8f4b-7f090aebbe76\") " pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117175 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f701165c-9e6c-4815-9bbc-f97ad6de524e-system-cni-dir\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117218 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-system-cni-dir\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117239 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-hostroot\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117261 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mgbj\" (UniqueName: \"kubernetes.io/projected/fdcc1ba3-fd82-4fcb-be4b-92340a548edd-kube-api-access-5mgbj\") pod \"node-resolver-dz5b9\" (UID: \"fdcc1ba3-fd82-4fcb-be4b-92340a548edd\") " pod="openshift-dns/node-resolver-dz5b9" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117294 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/86594764-14f9-43fb-a595-518d0c910187-ovnkube-script-lib\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117336 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t5c5\" (UniqueName: \"kubernetes.io/projected/6b50a281-2f0c-4b83-963b-8ebecce1eb14-kube-api-access-5t5c5\") pod \"network-metrics-daemon-98k9b\" (UID: \"6b50a281-2f0c-4b83-963b-8ebecce1eb14\") " pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:40.118024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117366 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f701165c-9e6c-4815-9bbc-f97ad6de524e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117390 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-systemd-units\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117418 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-node-log\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117445 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117470 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c055b3b3-1fac-45d3-82c1-a3e18bb068f8-konnectivity-ca\") pod \"konnectivity-agent-sc8gn\" (UID: \"c055b3b3-1fac-45d3-82c1-a3e18bb068f8\") " pod="kube-system/konnectivity-agent-sc8gn" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117581 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/67a6cbcf-d6c3-4b71-bed3-93e4935db1ec-iptables-alerter-script\") pod \"iptables-alerter-8m2bd\" (UID: \"67a6cbcf-d6c3-4b71-bed3-93e4935db1ec\") " pod="openshift-network-operator/iptables-alerter-8m2bd" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117599 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117658 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f701165c-9e6c-4815-9bbc-f97ad6de524e-cni-binary-copy\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117662 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-multus-socket-dir-parent\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117721 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f701165c-9e6c-4815-9bbc-f97ad6de524e-system-cni-dir\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117726 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-kubelet\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117771 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-system-cni-dir\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117774 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/86594764-14f9-43fb-a595-518d0c910187-env-overrides\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117803 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-hostroot\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117819 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-var-lib-cni-multus\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117866 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-lib-modules\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117898 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-var-lib-kubelet\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.118826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117911 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f701165c-9e6c-4815-9bbc-f97ad6de524e-os-release\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.117990 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f701165c-9e6c-4815-9bbc-f97ad6de524e-os-release\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.118508 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c055b3b3-1fac-45d3-82c1-a3e18bb068f8-konnectivity-ca\") pod \"konnectivity-agent-sc8gn\" (UID: \"c055b3b3-1fac-45d3-82c1-a3e18bb068f8\") " pod="kube-system/konnectivity-agent-sc8gn" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.118574 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-var-lib-kubelet\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.118592 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f701165c-9e6c-4815-9bbc-f97ad6de524e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.118635 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/be71c614-9b1c-4c09-9e28-8e9bb9021149-sys-fs\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.118661 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-os-release\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.118832 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f701165c-9e6c-4815-9bbc-f97ad6de524e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.118910 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-kubelet-config\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.118960 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-var-lib-cni-multus\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119178 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c055b3b3-1fac-45d3-82c1-a3e18bb068f8-agent-certs\") pod \"konnectivity-agent-sc8gn\" (UID: \"c055b3b3-1fac-45d3-82c1-a3e18bb068f8\") " pod="kube-system/konnectivity-agent-sc8gn" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119193 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-multus-daemon-config\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119181 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-lib-modules\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119216 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f701165c-9e6c-4815-9bbc-f97ad6de524e-cnibin\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119224 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-var-lib-kubelet\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119258 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-tmp\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119337 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-run-k8s-cni-cncf-io\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.119573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119380 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hqrq\" (UniqueName: \"kubernetes.io/projected/be71c614-9b1c-4c09-9e28-8e9bb9021149-kube-api-access-7hqrq\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119421 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-modprobe-d\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119446 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-sysconfig\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119449 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-tuned\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119472 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f701165c-9e6c-4815-9bbc-f97ad6de524e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119484 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f701165c-9e6c-4815-9bbc-f97ad6de524e-cnibin\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119519 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5p78\" (UniqueName: \"kubernetes.io/projected/f701165c-9e6c-4815-9bbc-f97ad6de524e-kube-api-access-n5p78\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119537 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-run-k8s-cni-cncf-io\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119547 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-run-multus-certs\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119534 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-sysconfig\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119638 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-cni-binary-copy\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119653 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-modprobe-d\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119705 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119776 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-dbus\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119826 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-sysctl-conf\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119895 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-host\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119894 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f701165c-9e6c-4815-9bbc-f97ad6de524e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.120208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119901 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-dbus\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:40.120905 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119929 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-host-run-multus-certs\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.120905 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.119949 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-host\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.120905 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.120006 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-etc-sysctl-conf\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.125730 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.124210 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldxzm\" (UniqueName: \"kubernetes.io/projected/c4b91bfd-8a80-4c53-bc6d-3d5717c557ac-kube-api-access-ldxzm\") pod \"tuned-h976w\" (UID: \"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac\") " pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.125730 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.124447 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bspsb\" (UniqueName: \"kubernetes.io/projected/67a6cbcf-d6c3-4b71-bed3-93e4935db1ec-kube-api-access-bspsb\") pod \"iptables-alerter-8m2bd\" (UID: \"67a6cbcf-d6c3-4b71-bed3-93e4935db1ec\") " pod="openshift-network-operator/iptables-alerter-8m2bd" Jul 07 14:26:40.125730 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.124482 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft6f9\" (UniqueName: \"kubernetes.io/projected/f029d40b-288d-4026-97a5-119ff263ed93-kube-api-access-ft6f9\") pod \"node-ca-mbx8j\" (UID: \"f029d40b-288d-4026-97a5-119ff263ed93\") " pod="openshift-image-registry/node-ca-mbx8j" Jul 07 14:26:40.125730 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.125600 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:26:40.125730 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.125622 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:26:40.125730 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.125641 2555 projected.go:196] Error preparing data for projected volume kube-api-access-ks79h for pod openshift-network-diagnostics/network-check-target-bt4tq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:40.125730 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.125700 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h podName:c7ab7625-0322-4b19-8f4b-7f090aebbe76 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:40.625676755 +0000 UTC m=+1.973730299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ks79h" (UniqueName: "kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h") pod "network-check-target-bt4tq" (UID: "c7ab7625-0322-4b19-8f4b-7f090aebbe76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:40.128539 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.128516 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mgbj\" (UniqueName: \"kubernetes.io/projected/fdcc1ba3-fd82-4fcb-be4b-92340a548edd-kube-api-access-5mgbj\") pod \"node-resolver-dz5b9\" (UID: \"fdcc1ba3-fd82-4fcb-be4b-92340a548edd\") " pod="openshift-dns/node-resolver-dz5b9" Jul 07 14:26:40.131617 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.131594 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t5c5\" (UniqueName: \"kubernetes.io/projected/6b50a281-2f0c-4b83-963b-8ebecce1eb14-kube-api-access-5t5c5\") pod \"network-metrics-daemon-98k9b\" (UID: \"6b50a281-2f0c-4b83-963b-8ebecce1eb14\") " pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:40.132202 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.132182 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4rv2\" (UniqueName: \"kubernetes.io/projected/501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb-kube-api-access-z4rv2\") pod \"multus-h7jl6\" (UID: \"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb\") " pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.132593 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.132570 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hqrq\" (UniqueName: \"kubernetes.io/projected/be71c614-9b1c-4c09-9e28-8e9bb9021149-kube-api-access-7hqrq\") pod \"aws-ebs-csi-driver-node-4ksrh\" (UID: \"be71c614-9b1c-4c09-9e28-8e9bb9021149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.133262 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.133246 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5p78\" (UniqueName: \"kubernetes.io/projected/f701165c-9e6c-4815-9bbc-f97ad6de524e-kube-api-access-n5p78\") pod \"multus-additional-cni-plugins-b9xjq\" (UID: \"f701165c-9e6c-4815-9bbc-f97ad6de524e\") " pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.134557 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.134543 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8m2bd" Jul 07 14:26:40.179379 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:40.179356 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a862d27acdd6adcd1a30a65484081bd.slice/crio-a9b4589d5bce723c9ad4bc6d3e8fe0c12359c6672f5eea303f54f5a814453efe WatchSource:0}: Error finding container a9b4589d5bce723c9ad4bc6d3e8fe0c12359c6672f5eea303f54f5a814453efe: Status 404 returned error can't find the container with id a9b4589d5bce723c9ad4bc6d3e8fe0c12359c6672f5eea303f54f5a814453efe Jul 07 14:26:40.184392 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.184373 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 14:26:40.221170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221146 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-cni-bin\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221285 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221179 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-run-ovn-kubernetes\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221285 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221195 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/86594764-14f9-43fb-a595-518d0c910187-ovn-node-metrics-cert\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221285 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221210 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q7sd\" (UniqueName: \"kubernetes.io/projected/86594764-14f9-43fb-a595-518d0c910187-kube-api-access-7q7sd\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221285 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221268 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-cni-bin\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221429 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221315 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-run-ovn-kubernetes\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221468 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221442 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-slash\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221502 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221473 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-run-openvswitch\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221502 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221491 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-run-netns\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221523 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-var-lib-openvswitch\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221535 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-slash\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221564 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-run-openvswitch\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221707 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221599 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-var-lib-openvswitch\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221707 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221621 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-run-netns\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221803 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221759 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-cni-netd\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221915 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221860 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-cni-netd\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.221915 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221895 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-run-ovn\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222004 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221976 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/86594764-14f9-43fb-a595-518d0c910187-ovnkube-script-lib\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222052 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222009 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-systemd-units\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222052 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222036 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-node-log\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222225 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.221995 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-run-ovn\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222225 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222064 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-kubelet\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222225 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222108 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-systemd-units\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222225 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222136 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-kubelet\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222225 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222125 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/86594764-14f9-43fb-a595-518d0c910187-env-overrides\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222225 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222166 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-node-log\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222225 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222202 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222485 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222234 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-run-systemd\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222485 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222256 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-etc-openvswitch\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222485 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222262 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222485 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222280 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/86594764-14f9-43fb-a595-518d0c910187-ovnkube-config\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222485 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222307 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-run-systemd\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222485 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222318 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-log-socket\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222485 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222341 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-etc-openvswitch\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222485 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222441 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/86594764-14f9-43fb-a595-518d0c910187-log-socket\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222847 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222583 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/86594764-14f9-43fb-a595-518d0c910187-ovnkube-script-lib\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222847 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222779 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/86594764-14f9-43fb-a595-518d0c910187-env-overrides\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.222952 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.222938 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/86594764-14f9-43fb-a595-518d0c910187-ovnkube-config\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.223798 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.223783 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/86594764-14f9-43fb-a595-518d0c910187-ovn-node-metrics-cert\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.224771 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:40.224752 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod831aa4a8e60ac1bafce8327f1e975c1a.slice/crio-036e0411dd7ae60c1700906e44d02025428619e57d9d11b02f0502060b4a41d1 WatchSource:0}: Error finding container 036e0411dd7ae60c1700906e44d02025428619e57d9d11b02f0502060b4a41d1: Status 404 returned error can't find the container with id 036e0411dd7ae60c1700906e44d02025428619e57d9d11b02f0502060b4a41d1 Jul 07 14:26:40.228847 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.228829 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q7sd\" (UniqueName: \"kubernetes.io/projected/86594764-14f9-43fb-a595-518d0c910187-kube-api-access-7q7sd\") pod \"ovnkube-node-9ml5j\" (UID: \"86594764-14f9-43fb-a595-518d0c910187\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.316574 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.316497 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" Jul 07 14:26:40.326735 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:40.326714 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe71c614_9b1c_4c09_9e28_8e9bb9021149.slice/crio-c7b3b94b0580fd99b180d9bc916a6b0ee520ade63f49a90cf3cfb96f550137ef WatchSource:0}: Error finding container c7b3b94b0580fd99b180d9bc916a6b0ee520ade63f49a90cf3cfb96f550137ef: Status 404 returned error can't find the container with id c7b3b94b0580fd99b180d9bc916a6b0ee520ade63f49a90cf3cfb96f550137ef Jul 07 14:26:40.332723 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.332707 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mbx8j" Jul 07 14:26:40.340489 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:40.340469 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf029d40b_288d_4026_97a5_119ff263ed93.slice/crio-73211f7d40d9e00d9dd6e6f052882ac8212334c7931479a037517fd5a65af960 WatchSource:0}: Error finding container 73211f7d40d9e00d9dd6e6f052882ac8212334c7931479a037517fd5a65af960: Status 404 returned error can't find the container with id 73211f7d40d9e00d9dd6e6f052882ac8212334c7931479a037517fd5a65af960 Jul 07 14:26:40.351811 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.351782 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sc8gn" Jul 07 14:26:40.360262 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:40.360242 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc055b3b3_1fac_45d3_82c1_a3e18bb068f8.slice/crio-38c2566131aa800ae95add0d89187fb6c3b05bf0fb98f3eade273cf4e1de9722 WatchSource:0}: Error finding container 38c2566131aa800ae95add0d89187fb6c3b05bf0fb98f3eade273cf4e1de9722: Status 404 returned error can't find the container with id 38c2566131aa800ae95add0d89187fb6c3b05bf0fb98f3eade273cf4e1de9722 Jul 07 14:26:40.368936 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.368917 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h976w" Jul 07 14:26:40.376878 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:40.376854 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b91bfd_8a80_4c53_bc6d_3d5717c557ac.slice/crio-24cc146a3274849251079889a7fac4d7ab3fca2cc0c6569074eba7f3d1a818d3 WatchSource:0}: Error finding container 24cc146a3274849251079889a7fac4d7ab3fca2cc0c6569074eba7f3d1a818d3: Status 404 returned error can't find the container with id 24cc146a3274849251079889a7fac4d7ab3fca2cc0c6569074eba7f3d1a818d3 Jul 07 14:26:40.386818 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.386800 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dz5b9" Jul 07 14:26:40.394114 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:40.394090 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdcc1ba3_fd82_4fcb_be4b_92340a548edd.slice/crio-0960db9cc27f00a91e6359d22f27d54f9be64f6980b1be7d84f697ed4ffa1536 WatchSource:0}: Error finding container 0960db9cc27f00a91e6359d22f27d54f9be64f6980b1be7d84f697ed4ffa1536: Status 404 returned error can't find the container with id 0960db9cc27f00a91e6359d22f27d54f9be64f6980b1be7d84f697ed4ffa1536 Jul 07 14:26:40.401657 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.401641 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b9xjq" Jul 07 14:26:40.408993 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:40.408974 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf701165c_9e6c_4815_9bbc_f97ad6de524e.slice/crio-28a634df9f4879dac75a91eb6a5e308aecdce90276850ffc40ae7f8f1d49b2a7 WatchSource:0}: Error finding container 28a634df9f4879dac75a91eb6a5e308aecdce90276850ffc40ae7f8f1d49b2a7: Status 404 returned error can't find the container with id 28a634df9f4879dac75a91eb6a5e308aecdce90276850ffc40ae7f8f1d49b2a7 Jul 07 14:26:40.414804 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.414783 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h7jl6" Jul 07 14:26:40.422303 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:40.422258 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod501dd9fb_4ed1_4bb2_b659_c4ad76ad83fb.slice/crio-7a2b14283b642e55998f106236448757fc19426faf996622052ab8ecb3f9d9f8 WatchSource:0}: Error finding container 7a2b14283b642e55998f106236448757fc19426faf996622052ab8ecb3f9d9f8: Status 404 returned error can't find the container with id 7a2b14283b642e55998f106236448757fc19426faf996622052ab8ecb3f9d9f8 Jul 07 14:26:40.439031 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.439003 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:26:40.447705 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:40.447687 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86594764_14f9_43fb_a595_518d0c910187.slice/crio-25ec380cc4239078e0e49e5459b682cce85c7689c11dd2ebfa0ceb51d13375f2 WatchSource:0}: Error finding container 25ec380cc4239078e0e49e5459b682cce85c7689c11dd2ebfa0ceb51d13375f2: Status 404 returned error can't find the container with id 25ec380cc4239078e0e49e5459b682cce85c7689c11dd2ebfa0ceb51d13375f2 Jul 07 14:26:40.625490 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.625373 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs\") pod \"network-metrics-daemon-98k9b\" (UID: \"6b50a281-2f0c-4b83-963b-8ebecce1eb14\") " pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:40.625490 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.625425 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:40.625674 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.625528 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:40.625674 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.625531 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:40.625674 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.625578 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret podName:cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:41.625565734 +0000 UTC m=+2.973619264 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret") pod "global-pull-secret-syncer-8c9rl" (UID: "cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:40.625674 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.625592 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs podName:6b50a281-2f0c-4b83-963b-8ebecce1eb14 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:41.625586446 +0000 UTC m=+2.973639977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs") pod "network-metrics-daemon-98k9b" (UID: "6b50a281-2f0c-4b83-963b-8ebecce1eb14") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:40.726200 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:40.726171 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks79h\" (UniqueName: \"kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h\") pod \"network-check-target-bt4tq\" (UID: \"c7ab7625-0322-4b19-8f4b-7f090aebbe76\") " pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:40.726353 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.726339 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:26:40.726392 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.726359 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:26:40.726392 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.726368 2555 projected.go:196] Error preparing data for projected volume kube-api-access-ks79h for pod openshift-network-diagnostics/network-check-target-bt4tq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:40.726446 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:40.726418 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h podName:c7ab7625-0322-4b19-8f4b-7f090aebbe76 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:41.726404721 +0000 UTC m=+3.074458252 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ks79h" (UniqueName: "kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h") pod "network-check-target-bt4tq" (UID: "c7ab7625-0322-4b19-8f4b-7f090aebbe76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:40.929870 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:26:40.929837 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67a6cbcf_d6c3_4b71_bed3_93e4935db1ec.slice/crio-dd6e6d3a8f2e6862698f64066fa11cf4c2f47a85fd207a2ec11063e66b48b7b5 WatchSource:0}: Error finding container dd6e6d3a8f2e6862698f64066fa11cf4c2f47a85fd207a2ec11063e66b48b7b5: Status 404 returned error can't find the container with id dd6e6d3a8f2e6862698f64066fa11cf4c2f47a85fd207a2ec11063e66b48b7b5 Jul 07 14:26:41.064941 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.064906 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:21:40 +0000 UTC" deadline="2028-01-04 02:05:00.365480893 +0000 UTC" Jul 07 14:26:41.064941 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.064940 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13091h38m19.30054551s" Jul 07 14:26:41.174011 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.173978 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:41.174206 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:41.174145 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:26:41.187607 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.186738 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9xjq" event={"ID":"f701165c-9e6c-4815-9bbc-f97ad6de524e","Type":"ContainerStarted","Data":"28a634df9f4879dac75a91eb6a5e308aecdce90276850ffc40ae7f8f1d49b2a7"} Jul 07 14:26:41.189790 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.189763 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dz5b9" event={"ID":"fdcc1ba3-fd82-4fcb-be4b-92340a548edd","Type":"ContainerStarted","Data":"0960db9cc27f00a91e6359d22f27d54f9be64f6980b1be7d84f697ed4ffa1536"} Jul 07 14:26:41.191563 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.191536 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sc8gn" event={"ID":"c055b3b3-1fac-45d3-82c1-a3e18bb068f8","Type":"ContainerStarted","Data":"38c2566131aa800ae95add0d89187fb6c3b05bf0fb98f3eade273cf4e1de9722"} Jul 07 14:26:41.193255 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.193231 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mbx8j" event={"ID":"f029d40b-288d-4026-97a5-119ff263ed93","Type":"ContainerStarted","Data":"73211f7d40d9e00d9dd6e6f052882ac8212334c7931479a037517fd5a65af960"} Jul 07 14:26:41.194649 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.194624 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8m2bd" event={"ID":"67a6cbcf-d6c3-4b71-bed3-93e4935db1ec","Type":"ContainerStarted","Data":"dd6e6d3a8f2e6862698f64066fa11cf4c2f47a85fd207a2ec11063e66b48b7b5"} Jul 07 14:26:41.195934 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.195909 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h7jl6" event={"ID":"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb","Type":"ContainerStarted","Data":"7a2b14283b642e55998f106236448757fc19426faf996622052ab8ecb3f9d9f8"} Jul 07 14:26:41.199054 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.199023 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h976w" event={"ID":"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac","Type":"ContainerStarted","Data":"24cc146a3274849251079889a7fac4d7ab3fca2cc0c6569074eba7f3d1a818d3"} Jul 07 14:26:41.202471 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.202421 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" event={"ID":"be71c614-9b1c-4c09-9e28-8e9bb9021149","Type":"ContainerStarted","Data":"c7b3b94b0580fd99b180d9bc916a6b0ee520ade63f49a90cf3cfb96f550137ef"} Jul 07 14:26:41.204854 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.204830 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal" event={"ID":"831aa4a8e60ac1bafce8327f1e975c1a","Type":"ContainerStarted","Data":"036e0411dd7ae60c1700906e44d02025428619e57d9d11b02f0502060b4a41d1"} Jul 07 14:26:41.207105 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.207082 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-129.ec2.internal" event={"ID":"4a862d27acdd6adcd1a30a65484081bd","Type":"ContainerStarted","Data":"a9b4589d5bce723c9ad4bc6d3e8fe0c12359c6672f5eea303f54f5a814453efe"} Jul 07 14:26:41.217792 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.217766 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" event={"ID":"86594764-14f9-43fb-a595-518d0c910187","Type":"ContainerStarted","Data":"25ec380cc4239078e0e49e5459b682cce85c7689c11dd2ebfa0ceb51d13375f2"} Jul 07 14:26:41.224380 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.224358 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:26:41.635211 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.634879 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:41.635422 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.635261 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs\") pod \"network-metrics-daemon-98k9b\" (UID: \"6b50a281-2f0c-4b83-963b-8ebecce1eb14\") " pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:41.635422 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:41.635159 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:41.635422 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:41.635423 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret podName:cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:43.635403782 +0000 UTC m=+4.983457330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret") pod "global-pull-secret-syncer-8c9rl" (UID: "cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:41.635422 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:41.635361 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:41.635673 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:41.635467 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs podName:6b50a281-2f0c-4b83-963b-8ebecce1eb14 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:43.635456582 +0000 UTC m=+4.983510114 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs") pod "network-metrics-daemon-98k9b" (UID: "6b50a281-2f0c-4b83-963b-8ebecce1eb14") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:41.736302 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:41.736263 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks79h\" (UniqueName: \"kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h\") pod \"network-check-target-bt4tq\" (UID: \"c7ab7625-0322-4b19-8f4b-7f090aebbe76\") " pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:41.736480 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:41.736452 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:26:41.736480 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:41.736475 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:26:41.736595 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:41.736487 2555 projected.go:196] Error preparing data for projected volume kube-api-access-ks79h for pod openshift-network-diagnostics/network-check-target-bt4tq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:41.736595 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:41.736544 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h podName:c7ab7625-0322-4b19-8f4b-7f090aebbe76 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:43.736524464 +0000 UTC m=+5.084577997 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ks79h" (UniqueName: "kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h") pod "network-check-target-bt4tq" (UID: "c7ab7625-0322-4b19-8f4b-7f090aebbe76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:42.023362 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:42.023318 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:26:42.066116 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:42.066029 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:21:40 +0000 UTC" deadline="2028-01-13 12:47:08.182047514 +0000 UTC" Jul 07 14:26:42.066116 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:42.066113 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13318h20m26.115939361s" Jul 07 14:26:42.173932 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:42.173899 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:42.174130 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:42.173936 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:42.174130 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:42.174038 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:26:42.174260 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:42.174166 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:26:43.033599 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:43.033565 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 14:26:43.187679 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:43.184471 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:43.187679 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:43.184580 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:26:43.656114 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:43.656061 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:43.656306 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:43.656150 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs\") pod \"network-metrics-daemon-98k9b\" (UID: \"6b50a281-2f0c-4b83-963b-8ebecce1eb14\") " pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:43.656306 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:43.656297 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:43.656423 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:43.656365 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs podName:6b50a281-2f0c-4b83-963b-8ebecce1eb14 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:47.656344107 +0000 UTC m=+9.004397640 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs") pod "network-metrics-daemon-98k9b" (UID: "6b50a281-2f0c-4b83-963b-8ebecce1eb14") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:43.656915 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:43.656721 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:43.656915 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:43.656794 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret podName:cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:47.656775238 +0000 UTC m=+9.004828769 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret") pod "global-pull-secret-syncer-8c9rl" (UID: "cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:43.757914 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:43.757193 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks79h\" (UniqueName: \"kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h\") pod \"network-check-target-bt4tq\" (UID: \"c7ab7625-0322-4b19-8f4b-7f090aebbe76\") " pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:43.757914 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:43.757420 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:26:43.757914 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:43.757437 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:26:43.757914 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:43.757449 2555 projected.go:196] Error preparing data for projected volume kube-api-access-ks79h for pod openshift-network-diagnostics/network-check-target-bt4tq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:43.757914 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:43.757507 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h podName:c7ab7625-0322-4b19-8f4b-7f090aebbe76 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:47.757489057 +0000 UTC m=+9.105542590 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ks79h" (UniqueName: "kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h") pod "network-check-target-bt4tq" (UID: "c7ab7625-0322-4b19-8f4b-7f090aebbe76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:44.174359 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:44.174328 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:44.174553 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:44.174468 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:26:44.174840 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:44.174820 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:44.174938 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:44.174917 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:26:45.178149 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:45.177659 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:45.178149 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:45.177781 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:26:46.174914 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:46.174212 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:46.174914 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:46.174358 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:26:46.174914 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:46.174774 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:46.174914 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:46.174860 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:26:47.176507 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:47.176476 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:47.176862 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:47.176601 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:26:47.692868 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:47.692825 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:47.693033 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:47.692898 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs\") pod \"network-metrics-daemon-98k9b\" (UID: \"6b50a281-2f0c-4b83-963b-8ebecce1eb14\") " pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:47.693033 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:47.693009 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:47.693179 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:47.693041 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:47.693179 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:47.693108 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret podName:cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:55.693085144 +0000 UTC m=+17.041138688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret") pod "global-pull-secret-syncer-8c9rl" (UID: "cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:47.693179 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:47.693131 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs podName:6b50a281-2f0c-4b83-963b-8ebecce1eb14 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:55.6931205 +0000 UTC m=+17.041174035 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs") pod "network-metrics-daemon-98k9b" (UID: "6b50a281-2f0c-4b83-963b-8ebecce1eb14") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:47.794125 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:47.794022 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks79h\" (UniqueName: \"kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h\") pod \"network-check-target-bt4tq\" (UID: \"c7ab7625-0322-4b19-8f4b-7f090aebbe76\") " pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:47.794321 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:47.794197 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:26:47.794321 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:47.794219 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:26:47.794321 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:47.794230 2555 projected.go:196] Error preparing data for projected volume kube-api-access-ks79h for pod openshift-network-diagnostics/network-check-target-bt4tq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:47.794321 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:47.794293 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h podName:c7ab7625-0322-4b19-8f4b-7f090aebbe76 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:55.794276339 +0000 UTC m=+17.142329870 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ks79h" (UniqueName: "kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h") pod "network-check-target-bt4tq" (UID: "c7ab7625-0322-4b19-8f4b-7f090aebbe76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:48.174248 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:48.174210 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:48.174438 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:48.174256 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:48.174438 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:48.174348 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:26:48.174547 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:48.174452 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:26:49.177694 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:49.176868 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:49.177694 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:49.176984 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:26:50.173624 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:50.173537 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:50.173802 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:50.173682 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:26:50.174041 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:50.173547 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:50.174182 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:50.174159 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:26:51.173621 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:51.173576 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:51.174120 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:51.173710 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:26:52.173508 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:52.173473 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:52.173752 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:52.173473 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:52.173752 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:52.173642 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:26:52.173752 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:52.173693 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:26:53.174492 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:53.174453 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:53.174902 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:53.174611 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:26:54.174269 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:54.174234 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:54.174442 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:54.174251 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:54.174442 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:54.174409 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:26:54.174604 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:54.174570 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:26:55.173871 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:55.173837 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:55.174052 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:55.173948 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:26:55.763742 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:55.763704 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:55.764224 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:55.763763 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs\") pod \"network-metrics-daemon-98k9b\" (UID: \"6b50a281-2f0c-4b83-963b-8ebecce1eb14\") " pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:55.764224 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:55.763857 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:55.764224 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:55.763872 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:55.764224 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:55.763931 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret podName:cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:11.763908609 +0000 UTC m=+33.111962143 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret") pod "global-pull-secret-syncer-8c9rl" (UID: "cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:26:55.764224 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:55.763950 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs podName:6b50a281-2f0c-4b83-963b-8ebecce1eb14 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:11.763942917 +0000 UTC m=+33.111996449 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs") pod "network-metrics-daemon-98k9b" (UID: "6b50a281-2f0c-4b83-963b-8ebecce1eb14") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:26:55.864629 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:55.864592 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks79h\" (UniqueName: \"kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h\") pod \"network-check-target-bt4tq\" (UID: \"c7ab7625-0322-4b19-8f4b-7f090aebbe76\") " pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:55.864801 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:55.864765 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:26:55.864801 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:55.864787 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:26:55.864801 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:55.864800 2555 projected.go:196] Error preparing data for projected volume kube-api-access-ks79h for pod openshift-network-diagnostics/network-check-target-bt4tq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:55.864954 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:55.864866 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h podName:c7ab7625-0322-4b19-8f4b-7f090aebbe76 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:11.864846761 +0000 UTC m=+33.212900297 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ks79h" (UniqueName: "kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h") pod "network-check-target-bt4tq" (UID: "c7ab7625-0322-4b19-8f4b-7f090aebbe76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:26:56.173711 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:56.173627 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:56.173878 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:56.173765 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:26:56.173878 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:56.173795 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:56.173878 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:56.173861 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:26:57.173648 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:57.173553 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:57.174102 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:57.173694 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:26:58.174009 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:58.173972 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:26:58.174464 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:58.174014 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:26:58.174464 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:58.174138 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:26:58.174464 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:58.174231 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:26:59.174735 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:26:59.174703 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:26:59.175171 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:26:59.174817 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:27:00.174326 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:00.174291 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:27:00.174326 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:00.174321 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:27:00.174549 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:00.174418 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:27:00.174594 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:00.174555 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:27:01.174565 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:01.174540 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:27:01.174891 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:01.174669 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:27:01.276293 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:01.276266 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h7jl6" event={"ID":"501dd9fb-4ed1-4bb2-b659-c4ad76ad83fb","Type":"ContainerStarted","Data":"a4b809a771c4d8163a10714450a15d5dbf15c2fccae3f7d3bf754235731f1c2d"} Jul 07 14:27:01.277928 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:01.277885 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h976w" event={"ID":"c4b91bfd-8a80-4c53-bc6d-3d5717c557ac","Type":"ContainerStarted","Data":"27c89a661927d80e09013e66c40c879788995fb31ab456616660a726f0d21945"} Jul 07 14:27:01.281465 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:01.281439 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-129.ec2.internal" event={"ID":"4a862d27acdd6adcd1a30a65484081bd","Type":"ContainerStarted","Data":"b6957cdd5f80334aae48c6cb62bb2c51cd20a2b807cfde479b3e9cf927fbb7b4"} Jul 07 14:27:01.297705 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:01.297466 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-h7jl6" podStartSLOduration=1.695300256 podStartE2EDuration="22.297449146s" totalImagesPullingTime="20.60214889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:01.297238909 +0000 UTC m=+22.645292462" watchObservedRunningTime="2026-07-07 14:27:01.297449146 +0000 UTC m=+22.645502700" Jul 07 14:27:01.310163 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:01.310123 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-129.ec2.internal" podStartSLOduration=21.310108037 podStartE2EDuration="21.310108037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:01.310107531 +0000 UTC m=+22.658161090" watchObservedRunningTime="2026-07-07 14:27:01.310108037 +0000 UTC m=+22.658161589" Jul 07 14:27:01.326739 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:01.326698 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h976w" podStartSLOduration=1.710726487 podStartE2EDuration="22.326686069s" totalImagesPullingTime="20.615959582s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:01.326219892 +0000 UTC m=+22.674273445" watchObservedRunningTime="2026-07-07 14:27:01.326686069 +0000 UTC m=+22.674739622" Jul 07 14:27:02.173988 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:02.173906 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:27:02.174181 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:02.173907 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:27:02.174181 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:02.174021 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:27:02.174181 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:02.174135 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:27:02.286028 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:02.286002 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:27:02.286734 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:02.286316 2555 generic.go:356] "Generic (PLEG): container finished" podID="86594764-14f9-43fb-a595-518d0c910187" containerID="f9f5aeef2213849c478d6a3e97c4f0196991febbe6daae9ceee7b30b7ec0b317" exitCode=1 Jul 07 14:27:02.288230 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:02.287122 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" event={"ID":"86594764-14f9-43fb-a595-518d0c910187","Type":"ContainerStarted","Data":"da2bc8bf0972a233abbc61337dbd1f143b0bc6492e1ed786b6e1244d995f8836"} Jul 07 14:27:02.288230 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:02.287286 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" event={"ID":"86594764-14f9-43fb-a595-518d0c910187","Type":"ContainerStarted","Data":"1c613a4b81b8eda3f6f117a6a51cee1b31ce076bac877efaa26ab8bb144055e8"} Jul 07 14:27:02.288230 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:02.287353 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" event={"ID":"86594764-14f9-43fb-a595-518d0c910187","Type":"ContainerStarted","Data":"bf4284ce3a18fa6af0c9b12ec269cf36b96b74b933845f11abfceb519ca7af61"} Jul 07 14:27:02.288230 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:02.287418 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" event={"ID":"86594764-14f9-43fb-a595-518d0c910187","Type":"ContainerStarted","Data":"21cbfc1d3c76d9d49986db65e0cbdd51cccda3c20c5c835b7f72b9d25f3d52f0"} Jul 07 14:27:02.288230 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:02.287481 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" event={"ID":"86594764-14f9-43fb-a595-518d0c910187","Type":"ContainerDied","Data":"f9f5aeef2213849c478d6a3e97c4f0196991febbe6daae9ceee7b30b7ec0b317"} Jul 07 14:27:02.288230 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:02.287544 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" event={"ID":"86594764-14f9-43fb-a595-518d0c910187","Type":"ContainerStarted","Data":"c8d31cf4c4990447ed9231eab196619bc4d1c648a53fc9af1310927c4819a278"} Jul 07 14:27:03.174293 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:03.174017 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:27:03.174634 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:03.174409 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:27:03.290214 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:03.290178 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8m2bd" event={"ID":"67a6cbcf-d6c3-4b71-bed3-93e4935db1ec","Type":"ContainerStarted","Data":"6391fd3dd76b8c398a99fbca1b6f3d72c855df113ab6275a8af147db106970e0"} Jul 07 14:27:03.292009 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:03.291978 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" event={"ID":"be71c614-9b1c-4c09-9e28-8e9bb9021149","Type":"ContainerStarted","Data":"3d948be044d8b9403d2e4f828e6a5c5c2bcd7b8fe79f1c7dd3edadb07424cd7e"} Jul 07 14:27:03.293698 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:03.293673 2555 generic.go:356] "Generic (PLEG): container finished" podID="831aa4a8e60ac1bafce8327f1e975c1a" containerID="4a80b4c006f4f496a318635fc487c0832c69e1802ec4abd684b359a7530a4e8a" exitCode=0 Jul 07 14:27:03.293838 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:03.293781 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal" event={"ID":"831aa4a8e60ac1bafce8327f1e975c1a","Type":"ContainerDied","Data":"4a80b4c006f4f496a318635fc487c0832c69e1802ec4abd684b359a7530a4e8a"} Jul 07 14:27:03.295539 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:03.295513 2555 generic.go:356] "Generic (PLEG): container finished" podID="f701165c-9e6c-4815-9bbc-f97ad6de524e" containerID="e617f19d566d502135e2199b75e44496d8b74c4f8ad4304920ff7fb7d26cc5ae" exitCode=0 Jul 07 14:27:03.295626 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:03.295594 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9xjq" event={"ID":"f701165c-9e6c-4815-9bbc-f97ad6de524e","Type":"ContainerDied","Data":"e617f19d566d502135e2199b75e44496d8b74c4f8ad4304920ff7fb7d26cc5ae"} Jul 07 14:27:03.297231 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:03.297209 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dz5b9" event={"ID":"fdcc1ba3-fd82-4fcb-be4b-92340a548edd","Type":"ContainerStarted","Data":"7112177e67bb041881ebfc0b6755c231ccb2bf88d9f10ab83122cd1083a88690"} Jul 07 14:27:03.298927 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:03.298879 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sc8gn" event={"ID":"c055b3b3-1fac-45d3-82c1-a3e18bb068f8","Type":"ContainerStarted","Data":"57a6d68c529c7b8939772aac9caa2b71deaa0ffa5c5789505937ce8478de9c3f"} Jul 07 14:27:03.300448 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:03.300428 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mbx8j" event={"ID":"f029d40b-288d-4026-97a5-119ff263ed93","Type":"ContainerStarted","Data":"e4a005e84ff0f4c90d5d02d17f1e6b573d3b86e276ec4a29e9c367d75f0e3962"} Jul 07 14:27:03.306005 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:03.305964 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8m2bd" podStartSLOduration=4.247420581 podStartE2EDuration="24.305953292s" totalImagesPullingTime="20.058532711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:03.305846179 +0000 UTC m=+24.653899732" watchObservedRunningTime="2026-07-07 14:27:03.305953292 +0000 UTC m=+24.654006856" Jul 07 14:27:03.338615 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:03.338565 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mbx8j" podStartSLOduration=12.266370603 podStartE2EDuration="24.338545516s" totalImagesPullingTime="12.072174913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:03.337928823 +0000 UTC m=+24.685982377" watchObservedRunningTime="2026-07-07 14:27:03.338545516 +0000 UTC m=+24.686599072" Jul 07 14:27:03.355125 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:03.355054 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sc8gn" podStartSLOduration=3.725247896 podStartE2EDuration="24.355040528s" totalImagesPullingTime="20.629792632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:03.355018023 +0000 UTC m=+24.703071586" watchObservedRunningTime="2026-07-07 14:27:03.355040528 +0000 UTC m=+24.703094100" Jul 07 14:27:03.384858 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:03.384817 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dz5b9" podStartSLOduration=3.805602405 podStartE2EDuration="24.384800874s" totalImagesPullingTime="20.579198469s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:03.384616481 +0000 UTC m=+24.732670035" watchObservedRunningTime="2026-07-07 14:27:03.384800874 +0000 UTC m=+24.732854428" Jul 07 14:27:03.808160 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:03.808136 2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 14:27:04.085351 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:04.085333 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mbx8j_f029d40b-288d-4026-97a5-119ff263ed93/node-ca/0.log" Jul 07 14:27:04.086439 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:04.086364 2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T14:27:03.808156556Z","UUID":"aca6f9bb-ac0a-41db-a516-b51b12e363a3","Handler":null,"Name":"","Endpoint":""} Jul 07 14:27:04.087711 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:04.087698 2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 14:27:04.087798 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:04.087723 2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 14:27:04.174274 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:04.174241 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:27:04.174425 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:04.174361 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:27:04.174737 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:04.174719 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:27:04.174846 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:04.174801 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:27:04.306668 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:04.306591 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:27:04.307120 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:04.307057 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" event={"ID":"86594764-14f9-43fb-a595-518d0c910187","Type":"ContainerStarted","Data":"a4714a7563aff6b666fd4b237084ce0aa556968aae4b5134f0a0aa5659b3e90f"} Jul 07 14:27:04.309257 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:04.309232 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" event={"ID":"be71c614-9b1c-4c09-9e28-8e9bb9021149","Type":"ContainerStarted","Data":"72f189d2a49a1e30700320cdc410a5ae29237975c6733126bc1526d2a38f7600"} Jul 07 14:27:04.311204 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:04.311166 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal" event={"ID":"831aa4a8e60ac1bafce8327f1e975c1a","Type":"ContainerStarted","Data":"1f53ab45b8b5934198204f4424c5d1574cc69361781ae4dab147b9a4326fb7a8"} Jul 07 14:27:04.328989 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:04.328938 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-129.ec2.internal" podStartSLOduration=24.328919831 podStartE2EDuration="24.328919831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:04.328216026 +0000 UTC m=+25.676269579" watchObservedRunningTime="2026-07-07 14:27:04.328919831 +0000 UTC m=+25.676973387" Jul 07 14:27:05.174439 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:05.173912 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:27:05.174439 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:05.174318 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:27:05.294789 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:05.294771 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sc8gn" Jul 07 14:27:05.295420 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:05.295397 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sc8gn" Jul 07 14:27:05.313612 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:05.313550 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sc8gn" Jul 07 14:27:05.314019 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:05.314004 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sc8gn" Jul 07 14:27:06.174453 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:06.174427 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:27:06.174563 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:06.174527 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:27:06.174563 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:06.174555 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:27:06.174627 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:06.174608 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:27:06.321205 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:06.320363 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" event={"ID":"be71c614-9b1c-4c09-9e28-8e9bb9021149","Type":"ContainerStarted","Data":"9c3545a8b6b288ebea49d76a894ef5d2b80cfe83c2a588b5698714282e31d134"} Jul 07 14:27:06.325677 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:06.325500 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:27:06.326058 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:06.325983 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" event={"ID":"86594764-14f9-43fb-a595-518d0c910187","Type":"ContainerStarted","Data":"e81c6faf2cb077a89abb7b32297646affd43520a4cada3f2a92b40bcfb4ba814"} Jul 07 14:27:06.326635 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:06.326473 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:27:06.326635 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:06.326548 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:27:06.326635 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:06.326614 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:27:06.326775 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:06.326550 2555 scope.go:117] "RemoveContainer" containerID="f9f5aeef2213849c478d6a3e97c4f0196991febbe6daae9ceee7b30b7ec0b317" Jul 07 14:27:06.340691 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:06.339140 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4ksrh" podStartSLOduration=3.9835458089999998 podStartE2EDuration="27.339122129s" totalImagesPullingTime="23.35557632s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:06.338624819 +0000 UTC m=+27.686678375" watchObservedRunningTime="2026-07-07 14:27:06.339122129 +0000 UTC m=+27.687175682" Jul 07 14:27:06.346509 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:06.344488 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:27:06.347858 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:06.346802 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:27:07.174665 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:07.174361 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:27:07.174806 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:07.174690 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:27:07.333151 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:07.333121 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:27:07.333658 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:07.333579 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" event={"ID":"86594764-14f9-43fb-a595-518d0c910187","Type":"ContainerStarted","Data":"255ac470c9e17eb7406b275e18a905d2000a58a27c0ac18f03b9ca22c9dad0b5"} Jul 07 14:27:07.364968 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:07.364927 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" podStartSLOduration=7.545258385 podStartE2EDuration="28.364912741s" totalImagesPullingTime="20.819654356s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:07.364435644 +0000 UTC m=+28.712489197" watchObservedRunningTime="2026-07-07 14:27:07.364912741 +0000 UTC m=+28.712966294" Jul 07 14:27:08.173870 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:08.173840 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:27:08.174045 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:08.173971 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:27:08.174045 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:08.174033 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:27:08.174197 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:08.174169 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:27:08.277037 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:08.277005 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8c9rl"] Jul 07 14:27:08.277214 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:08.277061 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bt4tq"] Jul 07 14:27:08.277255 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:08.277219 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:27:08.277312 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:08.277292 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:27:08.290657 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:08.290628 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-98k9b"] Jul 07 14:27:08.335965 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:08.335890 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:27:08.336406 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:08.335894 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:27:08.336406 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:08.336022 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:27:08.336406 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:08.336135 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:27:10.174053 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:10.173822 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:27:10.174413 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:10.173848 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:27:10.174413 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:10.174113 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:27:10.174413 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:10.173868 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:27:10.174413 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:10.174264 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:27:10.174413 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:10.174318 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:27:10.342599 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:10.342563 2555 generic.go:356] "Generic (PLEG): container finished" podID="f701165c-9e6c-4815-9bbc-f97ad6de524e" containerID="d0d539ca10d89a5b71b678433e860485d56ec68658c968d493b79862811e7673" exitCode=0 Jul 07 14:27:10.342743 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:10.342613 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9xjq" event={"ID":"f701165c-9e6c-4815-9bbc-f97ad6de524e","Type":"ContainerDied","Data":"d0d539ca10d89a5b71b678433e860485d56ec68658c968d493b79862811e7673"} Jul 07 14:27:11.347086 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:11.347031 2555 generic.go:356] "Generic (PLEG): container finished" podID="f701165c-9e6c-4815-9bbc-f97ad6de524e" containerID="7c35d8df23e0ed8d397ebe96122f9f9873775e8894e6dcf77df59b0ec15fc93b" exitCode=0 Jul 07 14:27:11.347563 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:11.347109 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9xjq" event={"ID":"f701165c-9e6c-4815-9bbc-f97ad6de524e","Type":"ContainerDied","Data":"7c35d8df23e0ed8d397ebe96122f9f9873775e8894e6dcf77df59b0ec15fc93b"} Jul 07 14:27:11.785312 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:11.785277 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs\") pod \"network-metrics-daemon-98k9b\" (UID: \"6b50a281-2f0c-4b83-963b-8ebecce1eb14\") " pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:27:11.785450 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:11.785337 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:27:11.785450 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:11.785419 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:27:11.785450 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:11.785425 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 14:27:11.785551 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:11.785480 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs podName:6b50a281-2f0c-4b83-963b-8ebecce1eb14 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:43.785464135 +0000 UTC m=+65.133517665 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs") pod "network-metrics-daemon-98k9b" (UID: "6b50a281-2f0c-4b83-963b-8ebecce1eb14") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 14:27:11.785551 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:11.785494 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret podName:cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:43.785488068 +0000 UTC m=+65.133541599 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret") pod "global-pull-secret-syncer-8c9rl" (UID: "cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6") : object "kube-system"/"original-pull-secret" not registered Jul 07 14:27:11.886358 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:11.886315 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks79h\" (UniqueName: \"kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h\") pod \"network-check-target-bt4tq\" (UID: \"c7ab7625-0322-4b19-8f4b-7f090aebbe76\") " pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:27:11.886501 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:11.886456 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 14:27:11.886501 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:11.886478 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 14:27:11.886501 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:11.886488 2555 projected.go:196] Error preparing data for projected volume kube-api-access-ks79h for pod openshift-network-diagnostics/network-check-target-bt4tq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:27:11.886606 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:11.886535 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h podName:c7ab7625-0322-4b19-8f4b-7f090aebbe76 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:43.886522182 +0000 UTC m=+65.234575713 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ks79h" (UniqueName: "kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h") pod "network-check-target-bt4tq" (UID: "c7ab7625-0322-4b19-8f4b-7f090aebbe76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 14:27:12.174348 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:12.174323 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:27:12.174443 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:12.174323 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:27:12.174443 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:12.174426 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:27:12.174522 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:12.174323 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:27:12.174522 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:12.174503 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:27:12.174587 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:12.174570 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:27:12.351220 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:12.351179 2555 generic.go:356] "Generic (PLEG): container finished" podID="f701165c-9e6c-4815-9bbc-f97ad6de524e" containerID="9ed6cd6d108e736c2c3cf7ef2a91b16675bd482187a0f1057d0de7edb21ddcc8" exitCode=0 Jul 07 14:27:12.351220 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:12.351225 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9xjq" event={"ID":"f701165c-9e6c-4815-9bbc-f97ad6de524e","Type":"ContainerDied","Data":"9ed6cd6d108e736c2c3cf7ef2a91b16675bd482187a0f1057d0de7edb21ddcc8"} Jul 07 14:27:14.174278 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:14.174245 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:27:14.174807 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:14.174345 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:27:14.174807 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:14.174356 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4tq" podUID="c7ab7625-0322-4b19-8f4b-7f090aebbe76" Jul 07 14:27:14.174807 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:14.174381 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:27:14.174807 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:14.174494 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98k9b" podUID="6b50a281-2f0c-4b83-963b-8ebecce1eb14" Jul 07 14:27:14.174807 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:14.174565 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8c9rl" podUID="cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6" Jul 07 14:27:14.982859 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:14.982835 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-129.ec2.internal" event="NodeReady" Jul 07 14:27:14.983032 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:14.982994 2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 14:27:15.037601 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.036742 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7r4qc"] Jul 07 14:27:15.038794 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.038770 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mjxmj"] Jul 07 14:27:15.038934 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.038912 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7r4qc" Jul 07 14:27:15.040865 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.040845 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.041216 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.041170 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 14:27:15.042343 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.042316 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 14:27:15.045117 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.043338 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ck6xv\"" Jul 07 14:27:15.045117 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.043440 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 14:27:15.045117 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.043560 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 14:27:15.045117 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.043727 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 14:27:15.045117 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.044508 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 14:27:15.045117 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.044831 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fh5qh\"" Jul 07 14:27:15.045117 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.044955 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 14:27:15.049805 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.049786 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7r4qc"] Jul 07 14:27:15.054304 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.054282 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mjxmj"] Jul 07 14:27:15.059008 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.058990 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b2slt"] Jul 07 14:27:15.060779 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.060759 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:15.064326 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.064309 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ksq8k\"" Jul 07 14:27:15.064626 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.064609 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 14:27:15.064759 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.064743 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 14:27:15.074094 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.074056 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b2slt"] Jul 07 14:27:15.211736 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.211700 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvf7h\" (UniqueName: \"kubernetes.io/projected/9413cf3e-94cd-4a72-9c07-495de8f1e33c-kube-api-access-pvf7h\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.212167 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.211757 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88fc15e7-95c6-453e-afa7-e9b3ae0384ec-cert\") pod \"ingress-canary-7r4qc\" (UID: \"88fc15e7-95c6-453e-afa7-e9b3ae0384ec\") " pod="openshift-ingress-canary/ingress-canary-7r4qc" Jul 07 14:27:15.212167 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.211808 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9413cf3e-94cd-4a72-9c07-495de8f1e33c-crio-socket\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.212167 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.211849 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vwf7\" (UniqueName: \"kubernetes.io/projected/88fc15e7-95c6-453e-afa7-e9b3ae0384ec-kube-api-access-5vwf7\") pod \"ingress-canary-7r4qc\" (UID: \"88fc15e7-95c6-453e-afa7-e9b3ae0384ec\") " pod="openshift-ingress-canary/ingress-canary-7r4qc" Jul 07 14:27:15.212167 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.211937 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9413cf3e-94cd-4a72-9c07-495de8f1e33c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.212167 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.211973 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.212167 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.211994 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b4dfc0ec-e629-439f-b444-1a6784f4f29b-config-volume\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:15.212167 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.212010 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5pbb\" (UniqueName: \"kubernetes.io/projected/b4dfc0ec-e629-439f-b444-1a6784f4f29b-kube-api-access-l5pbb\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:15.212167 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.212033 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc0ec-e629-439f-b444-1a6784f4f29b-tmp-dir\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:15.212167 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.212053 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9413cf3e-94cd-4a72-9c07-495de8f1e33c-data-volume\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.212167 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.212126 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:15.312902 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.312820 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vwf7\" (UniqueName: \"kubernetes.io/projected/88fc15e7-95c6-453e-afa7-e9b3ae0384ec-kube-api-access-5vwf7\") pod \"ingress-canary-7r4qc\" (UID: \"88fc15e7-95c6-453e-afa7-e9b3ae0384ec\") " pod="openshift-ingress-canary/ingress-canary-7r4qc" Jul 07 14:27:15.312902 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.312868 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9413cf3e-94cd-4a72-9c07-495de8f1e33c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.312902 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.312893 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.313182 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.313047 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b4dfc0ec-e629-439f-b444-1a6784f4f29b-config-volume\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:15.313232 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:15.313213 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:27:15.313876 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.313267 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5pbb\" (UniqueName: \"kubernetes.io/projected/b4dfc0ec-e629-439f-b444-1a6784f4f29b-kube-api-access-l5pbb\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:15.313876 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:15.313285 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls podName:9413cf3e-94cd-4a72-9c07-495de8f1e33c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:15.813264992 +0000 UTC m=+37.161318526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mjxmj" (UID: "9413cf3e-94cd-4a72-9c07-495de8f1e33c") : secret "insights-runtime-extractor-tls" not found Jul 07 14:27:15.313876 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.313322 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc0ec-e629-439f-b444-1a6784f4f29b-tmp-dir\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:15.313876 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.313353 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9413cf3e-94cd-4a72-9c07-495de8f1e33c-data-volume\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.313876 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.313393 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:15.313876 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.313449 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvf7h\" (UniqueName: \"kubernetes.io/projected/9413cf3e-94cd-4a72-9c07-495de8f1e33c-kube-api-access-pvf7h\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.313876 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.313494 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88fc15e7-95c6-453e-afa7-e9b3ae0384ec-cert\") pod \"ingress-canary-7r4qc\" (UID: \"88fc15e7-95c6-453e-afa7-e9b3ae0384ec\") " pod="openshift-ingress-canary/ingress-canary-7r4qc" Jul 07 14:27:15.313876 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.313539 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9413cf3e-94cd-4a72-9c07-495de8f1e33c-crio-socket\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.313876 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.313700 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9413cf3e-94cd-4a72-9c07-495de8f1e33c-data-volume\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.313876 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.313732 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b4dfc0ec-e629-439f-b444-1a6784f4f29b-config-volume\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:15.313876 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.313756 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9413cf3e-94cd-4a72-9c07-495de8f1e33c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.313876 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.313766 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4dfc0ec-e629-439f-b444-1a6784f4f29b-tmp-dir\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:15.313876 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:15.313777 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:27:15.313876 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:15.313811 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls podName:b4dfc0ec-e629-439f-b444-1a6784f4f29b nodeName:}" failed. No retries permitted until 2026-07-07 14:27:15.813799701 +0000 UTC m=+37.161853232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls") pod "dns-default-b2slt" (UID: "b4dfc0ec-e629-439f-b444-1a6784f4f29b") : secret "dns-default-metrics-tls" not found Jul 07 14:27:15.313876 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.313811 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9413cf3e-94cd-4a72-9c07-495de8f1e33c-crio-socket\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.318798 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.318777 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88fc15e7-95c6-453e-afa7-e9b3ae0384ec-cert\") pod \"ingress-canary-7r4qc\" (UID: \"88fc15e7-95c6-453e-afa7-e9b3ae0384ec\") " pod="openshift-ingress-canary/ingress-canary-7r4qc" Jul 07 14:27:15.324231 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.324203 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvf7h\" (UniqueName: \"kubernetes.io/projected/9413cf3e-94cd-4a72-9c07-495de8f1e33c-kube-api-access-pvf7h\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.324921 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.324896 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vwf7\" (UniqueName: \"kubernetes.io/projected/88fc15e7-95c6-453e-afa7-e9b3ae0384ec-kube-api-access-5vwf7\") pod \"ingress-canary-7r4qc\" (UID: \"88fc15e7-95c6-453e-afa7-e9b3ae0384ec\") " pod="openshift-ingress-canary/ingress-canary-7r4qc" Jul 07 14:27:15.325158 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.325131 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5pbb\" (UniqueName: \"kubernetes.io/projected/b4dfc0ec-e629-439f-b444-1a6784f4f29b-kube-api-access-l5pbb\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:15.356659 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.356633 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7r4qc" Jul 07 14:27:15.557034 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.556761 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7r4qc"] Jul 07 14:27:15.560546 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:27:15.560512 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88fc15e7_95c6_453e_afa7_e9b3ae0384ec.slice/crio-7a2c02947be9b08f0a1ec136f3c92c060fc6f5c2fb8c8190fe23656892375879 WatchSource:0}: Error finding container 7a2c02947be9b08f0a1ec136f3c92c060fc6f5c2fb8c8190fe23656892375879: Status 404 returned error can't find the container with id 7a2c02947be9b08f0a1ec136f3c92c060fc6f5c2fb8c8190fe23656892375879 Jul 07 14:27:15.817737 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.817635 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:15.817737 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:15.817699 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:15.817928 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:15.817817 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:27:15.817928 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:15.817845 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:27:15.817928 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:15.817903 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls podName:9413cf3e-94cd-4a72-9c07-495de8f1e33c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:16.817881731 +0000 UTC m=+38.165935276 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mjxmj" (UID: "9413cf3e-94cd-4a72-9c07-495de8f1e33c") : secret "insights-runtime-extractor-tls" not found Jul 07 14:27:15.817928 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:15.817921 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls podName:b4dfc0ec-e629-439f-b444-1a6784f4f29b nodeName:}" failed. No retries permitted until 2026-07-07 14:27:16.817909553 +0000 UTC m=+38.165963085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls") pod "dns-default-b2slt" (UID: "b4dfc0ec-e629-439f-b444-1a6784f4f29b") : secret "dns-default-metrics-tls" not found Jul 07 14:27:16.174244 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:16.174172 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:27:16.174447 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:16.174172 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:27:16.174447 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:16.174172 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:27:16.176787 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:16.176764 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 14:27:16.177423 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:16.177401 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 14:27:16.177541 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:16.177458 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-29x7z\"" Jul 07 14:27:16.177541 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:16.177498 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 14:27:16.177651 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:16.177405 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ps6h4\"" Jul 07 14:27:16.177651 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:16.177637 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 14:27:16.363204 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:16.363168 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7r4qc" event={"ID":"88fc15e7-95c6-453e-afa7-e9b3ae0384ec","Type":"ContainerStarted","Data":"7a2c02947be9b08f0a1ec136f3c92c060fc6f5c2fb8c8190fe23656892375879"} Jul 07 14:27:16.824708 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:16.824670 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:16.824881 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:16.824798 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:16.824881 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:16.824867 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:27:16.824973 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:16.824906 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:27:16.824973 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:16.824941 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls podName:b4dfc0ec-e629-439f-b444-1a6784f4f29b nodeName:}" failed. No retries permitted until 2026-07-07 14:27:18.82491964 +0000 UTC m=+40.172973190 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls") pod "dns-default-b2slt" (UID: "b4dfc0ec-e629-439f-b444-1a6784f4f29b") : secret "dns-default-metrics-tls" not found Jul 07 14:27:16.824973 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:16.824968 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls podName:9413cf3e-94cd-4a72-9c07-495de8f1e33c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:18.824949065 +0000 UTC m=+40.173002599 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mjxmj" (UID: "9413cf3e-94cd-4a72-9c07-495de8f1e33c") : secret "insights-runtime-extractor-tls" not found Jul 07 14:27:18.370092 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:18.370038 2555 generic.go:356] "Generic (PLEG): container finished" podID="f701165c-9e6c-4815-9bbc-f97ad6de524e" containerID="490c58c48f5c00a7e0081472ae6e9d983f2db4b50fade5a880eff752236e8fcc" exitCode=0 Jul 07 14:27:18.370447 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:18.370110 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9xjq" event={"ID":"f701165c-9e6c-4815-9bbc-f97ad6de524e","Type":"ContainerDied","Data":"490c58c48f5c00a7e0081472ae6e9d983f2db4b50fade5a880eff752236e8fcc"} Jul 07 14:27:18.840783 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:18.840746 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:18.840991 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:18.840901 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:27:18.841054 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:18.840902 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:18.841054 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:18.840981 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls podName:b4dfc0ec-e629-439f-b444-1a6784f4f29b nodeName:}" failed. No retries permitted until 2026-07-07 14:27:22.840958586 +0000 UTC m=+44.189012141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls") pod "dns-default-b2slt" (UID: "b4dfc0ec-e629-439f-b444-1a6784f4f29b") : secret "dns-default-metrics-tls" not found Jul 07 14:27:18.841054 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:18.840991 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:27:18.841198 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:18.841062 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls podName:9413cf3e-94cd-4a72-9c07-495de8f1e33c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:22.841044724 +0000 UTC m=+44.189098278 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mjxmj" (UID: "9413cf3e-94cd-4a72-9c07-495de8f1e33c") : secret "insights-runtime-extractor-tls" not found Jul 07 14:27:19.375738 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:19.375703 2555 generic.go:356] "Generic (PLEG): container finished" podID="f701165c-9e6c-4815-9bbc-f97ad6de524e" containerID="dcad4711aacb36cea085dea224af730153a664376dcf1ed2184a1d311070ec3d" exitCode=0 Jul 07 14:27:19.376195 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:19.375749 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9xjq" event={"ID":"f701165c-9e6c-4815-9bbc-f97ad6de524e","Type":"ContainerDied","Data":"dcad4711aacb36cea085dea224af730153a664376dcf1ed2184a1d311070ec3d"} Jul 07 14:27:20.117367 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.117344 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xzjs7"] Jul 07 14:27:20.132187 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.132159 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xzjs7"] Jul 07 14:27:20.132304 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.132269 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xzjs7" Jul 07 14:27:20.134234 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.134211 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 07 14:27:20.134492 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.134462 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 07 14:27:20.134584 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.134537 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 07 14:27:20.134643 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.134607 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-swm5r\"" Jul 07 14:27:20.134815 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.134799 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 07 14:27:20.148418 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.148398 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/dc7ff775-8cb0-472c-b714-68c79ccd877e-signing-key\") pod \"service-ca-758f599c5-xzjs7\" (UID: \"dc7ff775-8cb0-472c-b714-68c79ccd877e\") " pod="openshift-service-ca/service-ca-758f599c5-xzjs7" Jul 07 14:27:20.148509 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.148468 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz967\" (UniqueName: \"kubernetes.io/projected/dc7ff775-8cb0-472c-b714-68c79ccd877e-kube-api-access-kz967\") pod \"service-ca-758f599c5-xzjs7\" (UID: \"dc7ff775-8cb0-472c-b714-68c79ccd877e\") " pod="openshift-service-ca/service-ca-758f599c5-xzjs7" Jul 07 14:27:20.148568 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.148508 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/dc7ff775-8cb0-472c-b714-68c79ccd877e-signing-cabundle\") pod \"service-ca-758f599c5-xzjs7\" (UID: \"dc7ff775-8cb0-472c-b714-68c79ccd877e\") " pod="openshift-service-ca/service-ca-758f599c5-xzjs7" Jul 07 14:27:20.249037 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.249011 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/dc7ff775-8cb0-472c-b714-68c79ccd877e-signing-key\") pod \"service-ca-758f599c5-xzjs7\" (UID: \"dc7ff775-8cb0-472c-b714-68c79ccd877e\") " pod="openshift-service-ca/service-ca-758f599c5-xzjs7" Jul 07 14:27:20.249164 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.249119 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz967\" (UniqueName: \"kubernetes.io/projected/dc7ff775-8cb0-472c-b714-68c79ccd877e-kube-api-access-kz967\") pod \"service-ca-758f599c5-xzjs7\" (UID: \"dc7ff775-8cb0-472c-b714-68c79ccd877e\") " pod="openshift-service-ca/service-ca-758f599c5-xzjs7" Jul 07 14:27:20.249220 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.249163 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/dc7ff775-8cb0-472c-b714-68c79ccd877e-signing-cabundle\") pod \"service-ca-758f599c5-xzjs7\" (UID: \"dc7ff775-8cb0-472c-b714-68c79ccd877e\") " pod="openshift-service-ca/service-ca-758f599c5-xzjs7" Jul 07 14:27:20.249810 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.249783 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/dc7ff775-8cb0-472c-b714-68c79ccd877e-signing-cabundle\") pod \"service-ca-758f599c5-xzjs7\" (UID: \"dc7ff775-8cb0-472c-b714-68c79ccd877e\") " pod="openshift-service-ca/service-ca-758f599c5-xzjs7" Jul 07 14:27:20.253400 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.253376 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/dc7ff775-8cb0-472c-b714-68c79ccd877e-signing-key\") pod \"service-ca-758f599c5-xzjs7\" (UID: \"dc7ff775-8cb0-472c-b714-68c79ccd877e\") " pod="openshift-service-ca/service-ca-758f599c5-xzjs7" Jul 07 14:27:20.258204 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.258155 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz967\" (UniqueName: \"kubernetes.io/projected/dc7ff775-8cb0-472c-b714-68c79ccd877e-kube-api-access-kz967\") pod \"service-ca-758f599c5-xzjs7\" (UID: \"dc7ff775-8cb0-472c-b714-68c79ccd877e\") " pod="openshift-service-ca/service-ca-758f599c5-xzjs7" Jul 07 14:27:20.380445 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.380341 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7r4qc" event={"ID":"88fc15e7-95c6-453e-afa7-e9b3ae0384ec","Type":"ContainerStarted","Data":"b0a3fe39817a8f9c40465d995300e2153e65e1cb7ebd823c022e1099015a2324"} Jul 07 14:27:20.383900 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.383868 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b9xjq" event={"ID":"f701165c-9e6c-4815-9bbc-f97ad6de524e","Type":"ContainerStarted","Data":"083f5891f82b30245aac76da0e7401506294a234f4af96156be3c3df4ac6d71a"} Jul 07 14:27:20.414282 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.414234 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7r4qc" podStartSLOduration=0.858067843 podStartE2EDuration="5.414215307s" totalImagesPullingTime="4.556147464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:20.413901662 +0000 UTC m=+41.761955216" watchObservedRunningTime="2026-07-07 14:27:20.414215307 +0000 UTC m=+41.762268858" Jul 07 14:27:20.443761 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.443731 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xzjs7" Jul 07 14:27:20.453373 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.453330 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-b9xjq" podStartSLOduration=7.909070593 podStartE2EDuration="41.453315132s" totalImagesPullingTime="33.544244539s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:20.452779518 +0000 UTC m=+41.800833075" watchObservedRunningTime="2026-07-07 14:27:20.453315132 +0000 UTC m=+41.801368715" Jul 07 14:27:20.604327 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:20.604296 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xzjs7"] Jul 07 14:27:20.607389 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:27:20.607362 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc7ff775_8cb0_472c_b714_68c79ccd877e.slice/crio-88067597b0cd826dc17da7ef21e190fed90cf4e197a2fccd6477b41db4e0d0b7 WatchSource:0}: Error finding container 88067597b0cd826dc17da7ef21e190fed90cf4e197a2fccd6477b41db4e0d0b7: Status 404 returned error can't find the container with id 88067597b0cd826dc17da7ef21e190fed90cf4e197a2fccd6477b41db4e0d0b7 Jul 07 14:27:21.388463 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:21.388386 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xzjs7" event={"ID":"dc7ff775-8cb0-472c-b714-68c79ccd877e","Type":"ContainerStarted","Data":"88067597b0cd826dc17da7ef21e190fed90cf4e197a2fccd6477b41db4e0d0b7"} Jul 07 14:27:22.869388 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:22.869349 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:22.869879 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:22.869485 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:22.869879 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:22.869548 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 14:27:22.869879 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:22.869611 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 14:27:22.869879 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:22.869635 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls podName:b4dfc0ec-e629-439f-b444-1a6784f4f29b nodeName:}" failed. No retries permitted until 2026-07-07 14:27:30.869611787 +0000 UTC m=+52.217665335 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls") pod "dns-default-b2slt" (UID: "b4dfc0ec-e629-439f-b444-1a6784f4f29b") : secret "dns-default-metrics-tls" not found Jul 07 14:27:22.869879 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:27:22.869675 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls podName:9413cf3e-94cd-4a72-9c07-495de8f1e33c nodeName:}" failed. No retries permitted until 2026-07-07 14:27:30.869655835 +0000 UTC m=+52.217709390 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mjxmj" (UID: "9413cf3e-94cd-4a72-9c07-495de8f1e33c") : secret "insights-runtime-extractor-tls" not found Jul 07 14:27:24.399266 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:24.399231 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xzjs7" event={"ID":"dc7ff775-8cb0-472c-b714-68c79ccd877e","Type":"ContainerStarted","Data":"d9150eafd0e8c183605daa1ad519865dcf4e137b62a8b7f7b8cde3c2f675993e"} Jul 07 14:27:24.418253 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:24.418202 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-xzjs7" podStartSLOduration=1.6898469010000001 podStartE2EDuration="4.41818821s" totalImagesPullingTime="2.728341309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:24.417706848 +0000 UTC m=+45.765760412" watchObservedRunningTime="2026-07-07 14:27:24.41818821 +0000 UTC m=+45.766241846" Jul 07 14:27:30.935147 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:30.935103 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:30.935726 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:30.935227 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:30.937579 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:30.937555 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4dfc0ec-e629-439f-b444-1a6784f4f29b-metrics-tls\") pod \"dns-default-b2slt\" (UID: \"b4dfc0ec-e629-439f-b444-1a6784f4f29b\") " pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:30.937666 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:30.937634 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9413cf3e-94cd-4a72-9c07-495de8f1e33c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mjxmj\" (UID: \"9413cf3e-94cd-4a72-9c07-495de8f1e33c\") " pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:30.963714 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:30.963687 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mjxmj" Jul 07 14:27:30.971845 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:30.971818 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:31.114764 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:31.114732 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mjxmj"] Jul 07 14:27:31.134974 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:31.134947 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b2slt"] Jul 07 14:27:31.138187 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:27:31.138166 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4dfc0ec_e629_439f_b444_1a6784f4f29b.slice/crio-0109422c0b3b8dc3dc939de7f0822614667d35f223c630d06f8b6ea24d032db8 WatchSource:0}: Error finding container 0109422c0b3b8dc3dc939de7f0822614667d35f223c630d06f8b6ea24d032db8: Status 404 returned error can't find the container with id 0109422c0b3b8dc3dc939de7f0822614667d35f223c630d06f8b6ea24d032db8 Jul 07 14:27:31.421129 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:31.421098 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mjxmj" event={"ID":"9413cf3e-94cd-4a72-9c07-495de8f1e33c","Type":"ContainerStarted","Data":"1ef9154230e075af747779113066ee9f77624acfda28274de5e14347a9604449"} Jul 07 14:27:31.422250 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:31.422228 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b2slt" event={"ID":"b4dfc0ec-e629-439f-b444-1a6784f4f29b","Type":"ContainerStarted","Data":"0109422c0b3b8dc3dc939de7f0822614667d35f223c630d06f8b6ea24d032db8"} Jul 07 14:27:36.439466 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:36.439426 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mjxmj" event={"ID":"9413cf3e-94cd-4a72-9c07-495de8f1e33c","Type":"ContainerStarted","Data":"216358d615385e8f792d8b91142160ebe448007764b6331e57190cbe32956380"} Jul 07 14:27:36.439882 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:36.439502 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mjxmj" event={"ID":"9413cf3e-94cd-4a72-9c07-495de8f1e33c","Type":"ContainerStarted","Data":"cb9b3d4bbca9cb89b1fdd4eb559a853dfc0ddaca45fa4570c706575af7f4cba4"} Jul 07 14:27:36.441276 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:36.441243 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b2slt" event={"ID":"b4dfc0ec-e629-439f-b444-1a6784f4f29b","Type":"ContainerStarted","Data":"390c5e397628b8312f8b64aec2a7f606024219b50ad54ac36ed9742e6ebe22d0"} Jul 07 14:27:36.441389 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:36.441297 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b2slt" event={"ID":"b4dfc0ec-e629-439f-b444-1a6784f4f29b","Type":"ContainerStarted","Data":"9f47f5706a6f36d672479489d54878a4392b8b3d6e93163d329db16e63d12897"} Jul 07 14:27:36.441442 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:36.441428 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:36.459694 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:36.459638 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b2slt" podStartSLOduration=16.961421809 podStartE2EDuration="21.459622254s" totalImagesPullingTime="4.498200445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:36.459141092 +0000 UTC m=+57.807194644" watchObservedRunningTime="2026-07-07 14:27:36.459622254 +0000 UTC m=+57.807675823" Jul 07 14:27:38.347632 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:38.347600 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ml5j" Jul 07 14:27:39.454672 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:39.454630 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mjxmj" event={"ID":"9413cf3e-94cd-4a72-9c07-495de8f1e33c","Type":"ContainerStarted","Data":"bc087bfccd2ee8f46637bd267163ae87cb3b85a2477f45d585cf456740419334"} Jul 07 14:27:39.473823 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:39.473775 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mjxmj" podStartSLOduration=16.98712194 podStartE2EDuration="24.47375965s" totalImagesPullingTime="7.48663771s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:27:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:39.472713542 +0000 UTC m=+60.820767106" watchObservedRunningTime="2026-07-07 14:27:39.47375965 +0000 UTC m=+60.821813204" Jul 07 14:27:41.335255 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.335218 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6656b4db5b-9lzhp"] Jul 07 14:27:41.356566 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.356527 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf"] Jul 07 14:27:41.356748 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.356679 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6656b4db5b-9lzhp" Jul 07 14:27:41.359094 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.359053 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 07 14:27:41.359565 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.359547 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 14:27:41.359864 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.359843 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 14:27:41.359954 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.359843 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 14:27:41.364504 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.364485 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-lkrwh\"" Jul 07 14:27:41.370376 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.370355 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6656b4db5b-9lzhp"] Jul 07 14:27:41.370463 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.370382 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf"] Jul 07 14:27:41.370463 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.370456 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" Jul 07 14:27:41.374603 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.374587 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 07 14:27:41.422646 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.422611 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9sbk\" (UniqueName: \"kubernetes.io/projected/38b614d8-5908-4aaf-93dd-5d8c8c9be410-kube-api-access-s9sbk\") pod \"klusterlet-addon-workmgr-5cf7466c47-kr5sf\" (UID: \"38b614d8-5908-4aaf-93dd-5d8c8c9be410\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" Jul 07 14:27:41.422831 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.422703 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/38b614d8-5908-4aaf-93dd-5d8c8c9be410-klusterlet-config\") pod \"klusterlet-addon-workmgr-5cf7466c47-kr5sf\" (UID: \"38b614d8-5908-4aaf-93dd-5d8c8c9be410\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" Jul 07 14:27:41.422831 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.422728 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3379992d-ed0d-4873-8f27-8f6dbdaeed87-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6656b4db5b-9lzhp\" (UID: \"3379992d-ed0d-4873-8f27-8f6dbdaeed87\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6656b4db5b-9lzhp" Jul 07 14:27:41.422831 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.422757 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q96l\" (UniqueName: \"kubernetes.io/projected/3379992d-ed0d-4873-8f27-8f6dbdaeed87-kube-api-access-9q96l\") pod \"managed-serviceaccount-addon-agent-6656b4db5b-9lzhp\" (UID: \"3379992d-ed0d-4873-8f27-8f6dbdaeed87\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6656b4db5b-9lzhp" Jul 07 14:27:41.422831 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.422809 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38b614d8-5908-4aaf-93dd-5d8c8c9be410-tmp\") pod \"klusterlet-addon-workmgr-5cf7466c47-kr5sf\" (UID: \"38b614d8-5908-4aaf-93dd-5d8c8c9be410\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" Jul 07 14:27:41.435188 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.435159 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb"] Jul 07 14:27:41.459964 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.459934 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb"] Jul 07 14:27:41.460148 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.460021 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.464488 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.464460 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 14:27:41.464618 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.464556 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6pj2m\"" Jul 07 14:27:41.464618 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.464584 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 14:27:41.464838 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.464821 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 14:27:41.471982 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.471957 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 14:27:41.523509 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.523474 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-installation-pull-secrets\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.523665 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.523518 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3379992d-ed0d-4873-8f27-8f6dbdaeed87-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6656b4db5b-9lzhp\" (UID: \"3379992d-ed0d-4873-8f27-8f6dbdaeed87\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6656b4db5b-9lzhp" Jul 07 14:27:41.523665 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.523572 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q96l\" (UniqueName: \"kubernetes.io/projected/3379992d-ed0d-4873-8f27-8f6dbdaeed87-kube-api-access-9q96l\") pod \"managed-serviceaccount-addon-agent-6656b4db5b-9lzhp\" (UID: \"3379992d-ed0d-4873-8f27-8f6dbdaeed87\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6656b4db5b-9lzhp" Jul 07 14:27:41.523665 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.523631 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-image-registry-private-configuration\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.523665 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.523653 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-registry-certificates\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.523848 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.523683 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9sbk\" (UniqueName: \"kubernetes.io/projected/38b614d8-5908-4aaf-93dd-5d8c8c9be410-kube-api-access-s9sbk\") pod \"klusterlet-addon-workmgr-5cf7466c47-kr5sf\" (UID: \"38b614d8-5908-4aaf-93dd-5d8c8c9be410\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" Jul 07 14:27:41.523848 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.523708 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb2kl\" (UniqueName: \"kubernetes.io/projected/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-kube-api-access-fb2kl\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.523848 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.523767 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-registry-tls\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.523848 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.523787 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-bound-sa-token\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.524029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.523865 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/38b614d8-5908-4aaf-93dd-5d8c8c9be410-klusterlet-config\") pod \"klusterlet-addon-workmgr-5cf7466c47-kr5sf\" (UID: \"38b614d8-5908-4aaf-93dd-5d8c8c9be410\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" Jul 07 14:27:41.524029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.523903 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-ca-trust-extracted\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.524029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.523940 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38b614d8-5908-4aaf-93dd-5d8c8c9be410-tmp\") pod \"klusterlet-addon-workmgr-5cf7466c47-kr5sf\" (UID: \"38b614d8-5908-4aaf-93dd-5d8c8c9be410\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" Jul 07 14:27:41.524029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.523979 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-trusted-ca\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.524344 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.524327 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38b614d8-5908-4aaf-93dd-5d8c8c9be410-tmp\") pod \"klusterlet-addon-workmgr-5cf7466c47-kr5sf\" (UID: \"38b614d8-5908-4aaf-93dd-5d8c8c9be410\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" Jul 07 14:27:41.526253 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.526234 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3379992d-ed0d-4873-8f27-8f6dbdaeed87-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6656b4db5b-9lzhp\" (UID: \"3379992d-ed0d-4873-8f27-8f6dbdaeed87\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6656b4db5b-9lzhp" Jul 07 14:27:41.526354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.526275 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/38b614d8-5908-4aaf-93dd-5d8c8c9be410-klusterlet-config\") pod \"klusterlet-addon-workmgr-5cf7466c47-kr5sf\" (UID: \"38b614d8-5908-4aaf-93dd-5d8c8c9be410\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" Jul 07 14:27:41.535758 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.535735 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9sbk\" (UniqueName: \"kubernetes.io/projected/38b614d8-5908-4aaf-93dd-5d8c8c9be410-kube-api-access-s9sbk\") pod \"klusterlet-addon-workmgr-5cf7466c47-kr5sf\" (UID: \"38b614d8-5908-4aaf-93dd-5d8c8c9be410\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" Jul 07 14:27:41.537036 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.537018 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q96l\" (UniqueName: \"kubernetes.io/projected/3379992d-ed0d-4873-8f27-8f6dbdaeed87-kube-api-access-9q96l\") pod \"managed-serviceaccount-addon-agent-6656b4db5b-9lzhp\" (UID: \"3379992d-ed0d-4873-8f27-8f6dbdaeed87\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6656b4db5b-9lzhp" Jul 07 14:27:41.625302 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.625221 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-trusted-ca\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.625302 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.625257 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-installation-pull-secrets\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.625497 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.625317 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-image-registry-private-configuration\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.625497 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.625343 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-registry-certificates\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.625497 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.625368 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb2kl\" (UniqueName: \"kubernetes.io/projected/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-kube-api-access-fb2kl\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.629268 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.626240 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-registry-tls\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.629268 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.626294 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-bound-sa-token\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.629268 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.626339 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-ca-trust-extracted\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.629268 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.626817 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-ca-trust-extracted\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.629268 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.627101 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-trusted-ca\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.629268 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.627168 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-registry-certificates\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.629268 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.629133 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-installation-pull-secrets\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.629950 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.629927 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-registry-tls\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.632000 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.631974 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-image-registry-private-configuration\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.634509 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.634486 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb2kl\" (UniqueName: \"kubernetes.io/projected/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-kube-api-access-fb2kl\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.638276 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.638258 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3571b67-a8e9-49f3-a6ab-cae8dd8bb559-bound-sa-token\") pod \"image-registry-6ccd6cbfc5-7pvdb\" (UID: \"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559\") " pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.677931 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.677887 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6656b4db5b-9lzhp" Jul 07 14:27:41.686476 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.685115 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" Jul 07 14:27:41.769437 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.769404 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:41.826706 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.826661 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6656b4db5b-9lzhp"] Jul 07 14:27:41.833932 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:27:41.833887 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3379992d_ed0d_4873_8f27_8f6dbdaeed87.slice/crio-960fb4c9616bc84478444900c2472dafb5adcedfeec277a9b845d844d17b49ee WatchSource:0}: Error finding container 960fb4c9616bc84478444900c2472dafb5adcedfeec277a9b845d844d17b49ee: Status 404 returned error can't find the container with id 960fb4c9616bc84478444900c2472dafb5adcedfeec277a9b845d844d17b49ee Jul 07 14:27:41.847548 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.847484 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf"] Jul 07 14:27:41.850278 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:27:41.850234 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38b614d8_5908_4aaf_93dd_5d8c8c9be410.slice/crio-4e65d78849675e8f3268399400401aaec7a75bf0a6b12483c68a54481d492f5d WatchSource:0}: Error finding container 4e65d78849675e8f3268399400401aaec7a75bf0a6b12483c68a54481d492f5d: Status 404 returned error can't find the container with id 4e65d78849675e8f3268399400401aaec7a75bf0a6b12483c68a54481d492f5d Jul 07 14:27:41.908914 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:41.908846 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb"] Jul 07 14:27:41.911901 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:27:41.911874 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3571b67_a8e9_49f3_a6ab_cae8dd8bb559.slice/crio-a18474ead5fe42e5908e970030b466ebf1ba568a4dff3dc0016c85ab2f8f8137 WatchSource:0}: Error finding container a18474ead5fe42e5908e970030b466ebf1ba568a4dff3dc0016c85ab2f8f8137: Status 404 returned error can't find the container with id a18474ead5fe42e5908e970030b466ebf1ba568a4dff3dc0016c85ab2f8f8137 Jul 07 14:27:42.464692 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:42.464655 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" event={"ID":"38b614d8-5908-4aaf-93dd-5d8c8c9be410","Type":"ContainerStarted","Data":"4e65d78849675e8f3268399400401aaec7a75bf0a6b12483c68a54481d492f5d"} Jul 07 14:27:42.465978 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:42.465949 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" event={"ID":"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559","Type":"ContainerStarted","Data":"89895a8dfa3c2fb9db21b2032f7956b475f776bb85cd589ec4afa4010a0266c4"} Jul 07 14:27:42.466055 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:42.465989 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" event={"ID":"a3571b67-a8e9-49f3-a6ab-cae8dd8bb559","Type":"ContainerStarted","Data":"a18474ead5fe42e5908e970030b466ebf1ba568a4dff3dc0016c85ab2f8f8137"} Jul 07 14:27:42.466055 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:42.466042 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:27:42.467251 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:42.467223 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6656b4db5b-9lzhp" event={"ID":"3379992d-ed0d-4873-8f27-8f6dbdaeed87","Type":"ContainerStarted","Data":"960fb4c9616bc84478444900c2472dafb5adcedfeec277a9b845d844d17b49ee"} Jul 07 14:27:42.486032 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:42.485983 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" podStartSLOduration=1.48596938 podStartE2EDuration="1.48596938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:42.484909182 +0000 UTC m=+63.832962736" watchObservedRunningTime="2026-07-07 14:27:42.48596938 +0000 UTC m=+63.834022911" Jul 07 14:27:43.848052 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:43.848014 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:27:43.848520 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:43.848090 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs\") pod \"network-metrics-daemon-98k9b\" (UID: \"6b50a281-2f0c-4b83-963b-8ebecce1eb14\") " pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:27:43.850621 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:43.850597 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 14:27:43.850923 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:43.850843 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 14:27:43.861280 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:43.861255 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b50a281-2f0c-4b83-963b-8ebecce1eb14-metrics-certs\") pod \"network-metrics-daemon-98k9b\" (UID: \"6b50a281-2f0c-4b83-963b-8ebecce1eb14\") " pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:27:43.861280 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:43.861276 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6-original-pull-secret\") pod \"global-pull-secret-syncer-8c9rl\" (UID: \"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6\") " pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:27:43.949063 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:43.949021 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks79h\" (UniqueName: \"kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h\") pod \"network-check-target-bt4tq\" (UID: \"c7ab7625-0322-4b19-8f4b-7f090aebbe76\") " pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:27:43.951536 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:43.951347 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 14:27:43.962466 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:43.962286 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 14:27:43.973399 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:43.973373 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks79h\" (UniqueName: \"kubernetes.io/projected/c7ab7625-0322-4b19-8f4b-7f090aebbe76-kube-api-access-ks79h\") pod \"network-check-target-bt4tq\" (UID: \"c7ab7625-0322-4b19-8f4b-7f090aebbe76\") " pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:27:44.086421 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:44.086390 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8c9rl" Jul 07 14:27:44.097144 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:44.097118 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-29x7z\"" Jul 07 14:27:44.102284 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:44.102218 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ps6h4\"" Jul 07 14:27:44.105543 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:44.105308 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:27:44.110517 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:44.110497 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98k9b" Jul 07 14:27:44.728322 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:44.728248 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bt4tq"] Jul 07 14:27:44.732577 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:27:44.732542 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7ab7625_0322_4b19_8f4b_7f090aebbe76.slice/crio-ef6254ba1172b951a1672b1dbf481ab2da6038a40492c54b8b308e2bc610c616 WatchSource:0}: Error finding container ef6254ba1172b951a1672b1dbf481ab2da6038a40492c54b8b308e2bc610c616: Status 404 returned error can't find the container with id ef6254ba1172b951a1672b1dbf481ab2da6038a40492c54b8b308e2bc610c616 Jul 07 14:27:44.753455 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:44.753383 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8c9rl"] Jul 07 14:27:44.757044 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:27:44.757012 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb20bfdc_dd4a_4961_a3ce_07939d8cd7c6.slice/crio-85079e3e1876f06105b59857ac822b71196a57ea3acfc2801dba03b56d2b8e0d WatchSource:0}: Error finding container 85079e3e1876f06105b59857ac822b71196a57ea3acfc2801dba03b56d2b8e0d: Status 404 returned error can't find the container with id 85079e3e1876f06105b59857ac822b71196a57ea3acfc2801dba03b56d2b8e0d Jul 07 14:27:44.758235 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:44.758194 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-98k9b"] Jul 07 14:27:44.763534 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:27:44.763512 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b50a281_2f0c_4b83_963b_8ebecce1eb14.slice/crio-2b663ec224260233ce0f3d1ce70627ec000948c6b02cdb408896125f5f9931b2 WatchSource:0}: Error finding container 2b663ec224260233ce0f3d1ce70627ec000948c6b02cdb408896125f5f9931b2: Status 404 returned error can't find the container with id 2b663ec224260233ce0f3d1ce70627ec000948c6b02cdb408896125f5f9931b2 Jul 07 14:27:45.480229 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:45.480118 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bt4tq" event={"ID":"c7ab7625-0322-4b19-8f4b-7f090aebbe76","Type":"ContainerStarted","Data":"ef6254ba1172b951a1672b1dbf481ab2da6038a40492c54b8b308e2bc610c616"} Jul 07 14:27:45.481744 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:45.481715 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-98k9b" event={"ID":"6b50a281-2f0c-4b83-963b-8ebecce1eb14","Type":"ContainerStarted","Data":"2b663ec224260233ce0f3d1ce70627ec000948c6b02cdb408896125f5f9931b2"} Jul 07 14:27:45.483496 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:45.483450 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8c9rl" event={"ID":"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6","Type":"ContainerStarted","Data":"85079e3e1876f06105b59857ac822b71196a57ea3acfc2801dba03b56d2b8e0d"} Jul 07 14:27:45.485985 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:45.485173 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6656b4db5b-9lzhp" event={"ID":"3379992d-ed0d-4873-8f27-8f6dbdaeed87","Type":"ContainerStarted","Data":"8685dda5f9ade99121ab333b1e08facc0b3e914084b1cd3facad6e7f13ce7e9c"} Jul 07 14:27:45.502755 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:45.502688 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6656b4db5b-9lzhp" podStartSLOduration=1.7362569209999998 podStartE2EDuration="4.502670902s" totalImagesPullingTime="2.766413981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:45.501689819 +0000 UTC m=+66.849743373" watchObservedRunningTime="2026-07-07 14:27:45.502670902 +0000 UTC m=+66.850724456" Jul 07 14:27:46.448054 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:46.448021 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b2slt" Jul 07 14:27:47.495976 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:47.495577 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" event={"ID":"38b614d8-5908-4aaf-93dd-5d8c8c9be410","Type":"ContainerStarted","Data":"c59ea7c916635005835ae4e2764d6a5334a1980bde98adfb50475db6aff36350"} Jul 07 14:27:47.495976 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:47.495819 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" Jul 07 14:27:47.497994 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:47.497957 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-98k9b" event={"ID":"6b50a281-2f0c-4b83-963b-8ebecce1eb14","Type":"ContainerStarted","Data":"a3bab178d9d4abdacf8962de130c8ac170dc0e75563382aa27698831a721741c"} Jul 07 14:27:47.498122 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:47.498017 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-98k9b" event={"ID":"6b50a281-2f0c-4b83-963b-8ebecce1eb14","Type":"ContainerStarted","Data":"f4c4ad6cdbee0333d0d0403904fbeee456198138375b5b45452ad949814f05c6"} Jul 07 14:27:47.504938 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:47.504891 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" Jul 07 14:27:47.513410 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:47.512629 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cf7466c47-kr5sf" podStartSLOduration=1.475201548 podStartE2EDuration="6.512611203s" totalImagesPullingTime="5.037409655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:47.511312748 +0000 UTC m=+68.859366300" watchObservedRunningTime="2026-07-07 14:27:47.512611203 +0000 UTC m=+68.860664758" Jul 07 14:27:47.527568 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:47.527520 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-98k9b" podStartSLOduration=66.421032348 podStartE2EDuration="1m8.527501941s" totalImagesPullingTime="2.106469593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:47.526342325 +0000 UTC m=+68.874395869" watchObservedRunningTime="2026-07-07 14:27:47.527501941 +0000 UTC m=+68.875555494" Jul 07 14:27:50.510307 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:50.510261 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bt4tq" event={"ID":"c7ab7625-0322-4b19-8f4b-7f090aebbe76","Type":"ContainerStarted","Data":"564cbcbbcd78ed354f2c1b91e41b1219773bb22cd46b13d12c9710668b00b62d"} Jul 07 14:27:50.510707 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:50.510398 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:27:50.511924 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:50.511902 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8c9rl" event={"ID":"cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6","Type":"ContainerStarted","Data":"cd807facbfd083fdf70a02df12678b16856cf08d2780f739e7be7e818d86c15b"} Jul 07 14:27:50.527761 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:50.527721 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bt4tq" podStartSLOduration=66.310308218 podStartE2EDuration="1m11.527706658s" totalImagesPullingTime="5.21739844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:50.526590542 +0000 UTC m=+71.874644095" watchObservedRunningTime="2026-07-07 14:27:50.527706658 +0000 UTC m=+71.875760210" Jul 07 14:27:50.542976 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:27:50.542932 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8c9rl" podStartSLOduration=66.349798078 podStartE2EDuration="1m11.542917561s" totalImagesPullingTime="5.193119483s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:50.542193246 +0000 UTC m=+71.890246822" watchObservedRunningTime="2026-07-07 14:27:50.542917561 +0000 UTC m=+71.890971154" Jul 07 14:28:03.475843 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:03.475711 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6ccd6cbfc5-7pvdb" Jul 07 14:28:05.133302 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:05.133267 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-bbblk"] Jul 07 14:28:05.136081 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:05.136048 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-bbblk" Jul 07 14:28:05.138338 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:05.138318 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 14:28:05.138950 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:05.138933 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-6b45p\"" Jul 07 14:28:05.138950 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:05.138947 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 14:28:05.148936 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:05.148913 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-bbblk"] Jul 07 14:28:05.199672 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:05.199639 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdnlz\" (UniqueName: \"kubernetes.io/projected/b705da46-8252-44d5-ae93-2bb5f79b5b97-kube-api-access-qdnlz\") pod \"downloads-685bfb5f64-bbblk\" (UID: \"b705da46-8252-44d5-ae93-2bb5f79b5b97\") " pod="openshift-console/downloads-685bfb5f64-bbblk" Jul 07 14:28:05.300301 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:05.300269 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdnlz\" (UniqueName: \"kubernetes.io/projected/b705da46-8252-44d5-ae93-2bb5f79b5b97-kube-api-access-qdnlz\") pod \"downloads-685bfb5f64-bbblk\" (UID: \"b705da46-8252-44d5-ae93-2bb5f79b5b97\") " pod="openshift-console/downloads-685bfb5f64-bbblk" Jul 07 14:28:05.308098 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:05.308056 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdnlz\" (UniqueName: \"kubernetes.io/projected/b705da46-8252-44d5-ae93-2bb5f79b5b97-kube-api-access-qdnlz\") pod \"downloads-685bfb5f64-bbblk\" (UID: \"b705da46-8252-44d5-ae93-2bb5f79b5b97\") " pod="openshift-console/downloads-685bfb5f64-bbblk" Jul 07 14:28:05.445509 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:05.445406 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-bbblk" Jul 07 14:28:05.572549 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:05.572469 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-bbblk"] Jul 07 14:28:05.575845 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:28:05.575820 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb705da46_8252_44d5_ae93_2bb5f79b5b97.slice/crio-c257284d52b1ca1ad7bec009429bdfc2b98138dca3a296f3e51764d995bd836b WatchSource:0}: Error finding container c257284d52b1ca1ad7bec009429bdfc2b98138dca3a296f3e51764d995bd836b: Status 404 returned error can't find the container with id c257284d52b1ca1ad7bec009429bdfc2b98138dca3a296f3e51764d995bd836b Jul 07 14:28:06.567608 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:06.567575 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-bbblk" event={"ID":"b705da46-8252-44d5-ae93-2bb5f79b5b97","Type":"ContainerStarted","Data":"c257284d52b1ca1ad7bec009429bdfc2b98138dca3a296f3e51764d995bd836b"} Jul 07 14:28:14.719741 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.719710 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rzvgc"] Jul 07 14:28:14.728284 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.728260 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.731645 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.730878 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 14:28:14.731645 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.731188 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 14:28:14.731645 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.731266 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 14:28:14.731645 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.731478 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 14:28:14.731645 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.731519 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 14:28:14.732052 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.731663 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 14:28:14.732052 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.731889 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nnrwj\"" Jul 07 14:28:14.871088 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.871033 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fffeadca-9677-487f-a937-65985edbba4d-metrics-client-ca\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.871268 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.871097 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fffeadca-9677-487f-a937-65985edbba4d-node-exporter-wtmp\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.871268 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.871156 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fffeadca-9677-487f-a937-65985edbba4d-root\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.871268 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.871188 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fffeadca-9677-487f-a937-65985edbba4d-sys\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.871268 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.871242 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fffeadca-9677-487f-a937-65985edbba4d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.871456 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.871291 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pkfr\" (UniqueName: \"kubernetes.io/projected/fffeadca-9677-487f-a937-65985edbba4d-kube-api-access-4pkfr\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.871456 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.871381 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fffeadca-9677-487f-a937-65985edbba4d-node-exporter-tls\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.871456 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.871419 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fffeadca-9677-487f-a937-65985edbba4d-node-exporter-accelerators-collector-config\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.871456 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.871443 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fffeadca-9677-487f-a937-65985edbba4d-node-exporter-textfile\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.972346 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.972258 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fffeadca-9677-487f-a937-65985edbba4d-node-exporter-wtmp\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.972346 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.972301 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fffeadca-9677-487f-a937-65985edbba4d-root\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.972560 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.972424 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fffeadca-9677-487f-a937-65985edbba4d-sys\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.972560 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.972453 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fffeadca-9677-487f-a937-65985edbba4d-root\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.972560 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.972472 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fffeadca-9677-487f-a937-65985edbba4d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.972560 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.972505 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pkfr\" (UniqueName: \"kubernetes.io/projected/fffeadca-9677-487f-a937-65985edbba4d-kube-api-access-4pkfr\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.972560 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.972516 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fffeadca-9677-487f-a937-65985edbba4d-sys\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.972560 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.972453 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fffeadca-9677-487f-a937-65985edbba4d-node-exporter-wtmp\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.972837 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.972620 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fffeadca-9677-487f-a937-65985edbba4d-node-exporter-tls\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.972837 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.972665 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fffeadca-9677-487f-a937-65985edbba4d-node-exporter-accelerators-collector-config\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.972837 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.972706 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fffeadca-9677-487f-a937-65985edbba4d-node-exporter-textfile\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.972837 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.972755 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fffeadca-9677-487f-a937-65985edbba4d-metrics-client-ca\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.973333 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.973308 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fffeadca-9677-487f-a937-65985edbba4d-node-exporter-accelerators-collector-config\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.973333 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.973323 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fffeadca-9677-487f-a937-65985edbba4d-node-exporter-textfile\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.973496 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.973368 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fffeadca-9677-487f-a937-65985edbba4d-metrics-client-ca\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.975304 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.975279 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fffeadca-9677-487f-a937-65985edbba4d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.975578 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.975558 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fffeadca-9677-487f-a937-65985edbba4d-node-exporter-tls\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:14.989936 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:14.989907 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pkfr\" (UniqueName: \"kubernetes.io/projected/fffeadca-9677-487f-a937-65985edbba4d-kube-api-access-4pkfr\") pod \"node-exporter-rzvgc\" (UID: \"fffeadca-9677-487f-a937-65985edbba4d\") " pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:15.040684 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.040651 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rzvgc" Jul 07 14:28:15.051775 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:28:15.051749 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfffeadca_9677_487f_a937_65985edbba4d.slice/crio-385ddaaa4c7a53bbf527f4aa87c81bb5101b136167faf163a8134bd23081985e WatchSource:0}: Error finding container 385ddaaa4c7a53bbf527f4aa87c81bb5101b136167faf163a8134bd23081985e: Status 404 returned error can't find the container with id 385ddaaa4c7a53bbf527f4aa87c81bb5101b136167faf163a8134bd23081985e Jul 07 14:28:15.600779 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.600743 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rzvgc" event={"ID":"fffeadca-9677-487f-a937-65985edbba4d","Type":"ContainerStarted","Data":"385ddaaa4c7a53bbf527f4aa87c81bb5101b136167faf163a8134bd23081985e"} Jul 07 14:28:15.806545 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.806510 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 14:28:15.826083 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.826033 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:15.828347 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.828320 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 14:28:15.829394 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.829333 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 14:28:15.829631 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.829333 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 14:28:15.829712 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.829697 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 14:28:15.830222 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.829890 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 14:28:15.830222 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.830021 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 14:28:15.830222 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.830104 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 14:28:15.830531 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.830242 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-k2q8h\"" Jul 07 14:28:15.830862 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.830843 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 14:28:15.833738 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.833714 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 14:28:15.835744 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.835674 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 14:28:15.979361 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.979322 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:15.979556 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.979375 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:15.979556 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.979415 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:15.979556 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.979447 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:15.979556 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.979477 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:15.979556 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.979503 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:15.979556 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.979528 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-config-volume\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:15.979556 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.979552 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:15.979912 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.979598 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75j46\" (UniqueName: \"kubernetes.io/projected/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-kube-api-access-75j46\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:15.979912 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.979675 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-web-config\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:15.979912 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.979712 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:15.979912 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.979744 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-config-out\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:15.979912 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:15.979780 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.080937 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.080902 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.081141 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.080952 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.081141 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.081091 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-config-volume\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.081141 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.081135 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.081304 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.081173 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75j46\" (UniqueName: \"kubernetes.io/projected/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-kube-api-access-75j46\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.081304 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.081219 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-web-config\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.081304 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.081238 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.081304 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.081258 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-config-out\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.081304 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.081281 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.081892 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.081606 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.081892 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.081645 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.081892 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.081697 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.081892 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.081734 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.082172 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.081974 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.082938 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.082598 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.085082 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.084350 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.085082 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.084584 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-web-config\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.085082 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.084934 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.085082 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.084964 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.085737 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.085694 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-config-volume\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.086356 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.086334 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.086596 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.086575 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.087155 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.087061 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.087266 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.087199 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.087753 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.087720 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-config-out\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.089827 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.089804 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75j46\" (UniqueName: \"kubernetes.io/projected/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-kube-api-access-75j46\") pod \"alertmanager-main-0\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.140051 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.140012 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:28:16.602700 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.602520 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 14:28:16.605398 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:28:16.605368 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e3bdde3_1587_4ca4_bc7a_2e0718ae0d02.slice/crio-c66ad2e6a8da5acba908710896af024d7bcde5a5732391022627fb7e61f7d77c WatchSource:0}: Error finding container c66ad2e6a8da5acba908710896af024d7bcde5a5732391022627fb7e61f7d77c: Status 404 returned error can't find the container with id c66ad2e6a8da5acba908710896af024d7bcde5a5732391022627fb7e61f7d77c Jul 07 14:28:16.793131 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.793098 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-65fdbf6b89-sfgth"] Jul 07 14:28:16.817607 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.817574 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65fdbf6b89-sfgth"] Jul 07 14:28:16.818248 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.818228 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:16.821198 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.821177 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 14:28:16.821347 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.821315 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 14:28:16.821424 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.821355 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-pzsmk\"" Jul 07 14:28:16.821537 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.821519 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 14:28:16.821641 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.821568 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 14:28:16.821641 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.821270 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ajp59jthq2p7e\"" Jul 07 14:28:16.821734 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.821210 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 14:28:16.990399 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.990286 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:16.990399 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.990333 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:16.990399 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.990364 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-grpc-tls\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:16.990666 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.990446 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:16.990666 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.990493 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b13d5133-cbd7-4d28-8f88-ea22a4a14014-metrics-client-ca\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:16.990666 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.990529 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r5g4\" (UniqueName: \"kubernetes.io/projected/b13d5133-cbd7-4d28-8f88-ea22a4a14014-kube-api-access-4r5g4\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:16.990666 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.990567 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:16.990666 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:16.990596 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-thanos-querier-tls\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.091304 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.091265 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.091304 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.091309 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-thanos-querier-tls\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.091492 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.091328 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.091492 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.091365 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.091492 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.091391 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-grpc-tls\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.091923 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.091735 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.091923 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.091814 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b13d5133-cbd7-4d28-8f88-ea22a4a14014-metrics-client-ca\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.092264 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.092175 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r5g4\" (UniqueName: \"kubernetes.io/projected/b13d5133-cbd7-4d28-8f88-ea22a4a14014-kube-api-access-4r5g4\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.092817 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.092759 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b13d5133-cbd7-4d28-8f88-ea22a4a14014-metrics-client-ca\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.094449 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.094402 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-thanos-querier-tls\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.094556 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.094473 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.094612 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.094577 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-grpc-tls\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.094769 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.094745 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.094957 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.094939 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.095224 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.095202 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b13d5133-cbd7-4d28-8f88-ea22a4a14014-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.112190 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.112166 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r5g4\" (UniqueName: \"kubernetes.io/projected/b13d5133-cbd7-4d28-8f88-ea22a4a14014-kube-api-access-4r5g4\") pod \"thanos-querier-65fdbf6b89-sfgth\" (UID: \"b13d5133-cbd7-4d28-8f88-ea22a4a14014\") " pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.130282 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.130264 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:17.294815 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.294780 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65fdbf6b89-sfgth"] Jul 07 14:28:17.298608 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:28:17.298568 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb13d5133_cbd7_4d28_8f88_ea22a4a14014.slice/crio-953f6389c51ef664c30dad9f031e38d9459de4e4a25b18a0beea2a0e70dc9aac WatchSource:0}: Error finding container 953f6389c51ef664c30dad9f031e38d9459de4e4a25b18a0beea2a0e70dc9aac: Status 404 returned error can't find the container with id 953f6389c51ef664c30dad9f031e38d9459de4e4a25b18a0beea2a0e70dc9aac Jul 07 14:28:17.610896 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.610815 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" event={"ID":"b13d5133-cbd7-4d28-8f88-ea22a4a14014","Type":"ContainerStarted","Data":"953f6389c51ef664c30dad9f031e38d9459de4e4a25b18a0beea2a0e70dc9aac"} Jul 07 14:28:17.612868 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.612839 2555 generic.go:356] "Generic (PLEG): container finished" podID="fffeadca-9677-487f-a937-65985edbba4d" containerID="a09321f6d1f259029a7334702265c358e3763dc3a7a17b47fb1beca836dc3f90" exitCode=0 Jul 07 14:28:17.613003 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.612933 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rzvgc" event={"ID":"fffeadca-9677-487f-a937-65985edbba4d","Type":"ContainerDied","Data":"a09321f6d1f259029a7334702265c358e3763dc3a7a17b47fb1beca836dc3f90"} Jul 07 14:28:17.614220 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:17.614197 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02","Type":"ContainerStarted","Data":"c66ad2e6a8da5acba908710896af024d7bcde5a5732391022627fb7e61f7d77c"} Jul 07 14:28:18.620791 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:18.620753 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rzvgc" event={"ID":"fffeadca-9677-487f-a937-65985edbba4d","Type":"ContainerStarted","Data":"c62399dbc220ce9264ebeca8894fec60b50fd54b6c81b5e59779a457ff2a8b66"} Jul 07 14:28:18.621223 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:18.620826 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rzvgc" event={"ID":"fffeadca-9677-487f-a937-65985edbba4d","Type":"ContainerStarted","Data":"920b8e9f5446354329a8276a684cfb27257272dfc9b34eda2548aee807987bea"} Jul 07 14:28:18.641305 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:18.641248 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rzvgc" podStartSLOduration=2.78688073 podStartE2EDuration="4.641228286s" totalImagesPullingTime="1.745358197s" totalInitContainerRuntime="108.989359ms" isStatefulPod=true podCreationTimestamp="2026-07-07 14:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:18.640105611 +0000 UTC m=+99.988159168" watchObservedRunningTime="2026-07-07 14:28:18.641228286 +0000 UTC m=+99.989281840" Jul 07 14:28:20.971763 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:20.971734 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 14:28:21.004908 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.004878 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 14:28:21.005112 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.005027 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.007817 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.007770 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 14:28:21.007817 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.007782 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 14:28:21.008052 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.008012 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 14:28:21.008052 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.008047 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 14:28:21.008228 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.008181 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 14:28:21.008228 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.008194 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7vvq35ka0u0ds\"" Jul 07 14:28:21.008358 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.008346 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 14:28:21.008439 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.008355 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kjzlf\"" Jul 07 14:28:21.008439 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.008376 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 14:28:21.008635 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.008615 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 14:28:21.008766 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.008740 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 14:28:21.008847 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.008829 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 14:28:21.008905 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.008873 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 14:28:21.009483 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.009121 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 14:28:21.009483 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.009207 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 14:28:21.011289 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.011268 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 14:28:21.015491 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.015472 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 14:28:21.129151 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129115 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129327 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129166 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-web-config\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129327 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129194 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129327 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129229 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129327 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129258 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbd86\" (UniqueName: \"kubernetes.io/projected/3e300535-55b4-4e26-ab19-336e55d21a7e-kube-api-access-tbd86\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129501 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129359 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e300535-55b4-4e26-ab19-336e55d21a7e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129501 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129391 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e300535-55b4-4e26-ab19-336e55d21a7e-config-out\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129501 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129411 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129501 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129440 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129501 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129471 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129501 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129493 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-config\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129721 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129524 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129721 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129593 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129721 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129621 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129721 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129654 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129721 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129705 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129939 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129737 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129939 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129752 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129939 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129772 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.129939 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.129896 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231305 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231222 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231305 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231267 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231305 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231290 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231558 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231343 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231558 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231370 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231558 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231400 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-web-config\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231558 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231422 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231558 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231452 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231558 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231478 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbd86\" (UniqueName: \"kubernetes.io/projected/3e300535-55b4-4e26-ab19-336e55d21a7e-kube-api-access-tbd86\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231558 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231517 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e300535-55b4-4e26-ab19-336e55d21a7e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231558 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231540 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e300535-55b4-4e26-ab19-336e55d21a7e-config-out\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231931 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231565 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231931 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231590 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231931 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231619 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231931 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231648 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-config\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231931 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231674 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231931 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231694 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231931 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231703 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231931 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231728 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231931 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231762 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.231931 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.231788 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.232499 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.232472 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.232623 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.232557 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.233147 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.233126 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.233271 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.233249 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.233501 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.233476 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.235783 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.235634 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.235783 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.235770 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e300535-55b4-4e26-ab19-336e55d21a7e-config-out\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.236654 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.236160 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.236654 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.236315 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.236654 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.236580 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.236654 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.236600 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.236910 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.236712 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.237558 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.237447 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.238281 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.238258 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-web-config\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.239020 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.238996 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e300535-55b4-4e26-ab19-336e55d21a7e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.239222 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.239201 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-config\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.239330 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.239307 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.239760 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.239735 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.240275 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.240256 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbd86\" (UniqueName: \"kubernetes.io/projected/3e300535-55b4-4e26-ab19-336e55d21a7e-kube-api-access-tbd86\") pod \"prometheus-k8s-0\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.317643 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.317606 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:28:21.518500 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:21.518417 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bt4tq" Jul 07 14:28:26.137133 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:26.137111 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 14:28:26.143527 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:28:26.143502 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e300535_55b4_4e26_ab19_336e55d21a7e.slice/crio-bbeaa3c3b37c4b0dba86ec41da4b13c7ad7964af82451ec193791c472363c0d7 WatchSource:0}: Error finding container bbeaa3c3b37c4b0dba86ec41da4b13c7ad7964af82451ec193791c472363c0d7: Status 404 returned error can't find the container with id bbeaa3c3b37c4b0dba86ec41da4b13c7ad7964af82451ec193791c472363c0d7 Jul 07 14:28:26.655780 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:26.655735 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-bbblk" event={"ID":"b705da46-8252-44d5-ae93-2bb5f79b5b97","Type":"ContainerStarted","Data":"a4c039e2fca03443c099a66ac47b9c6e19e12204ad6aa8e00050cfb84f1f00db"} Jul 07 14:28:26.655973 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:26.655926 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-bbblk" Jul 07 14:28:26.658789 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:26.658761 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" event={"ID":"b13d5133-cbd7-4d28-8f88-ea22a4a14014","Type":"ContainerStarted","Data":"5fdba6679597d83f10d232791ee2191e3d11abca685810ef838761b0c33e828e"} Jul 07 14:28:26.658912 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:26.658827 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" event={"ID":"b13d5133-cbd7-4d28-8f88-ea22a4a14014","Type":"ContainerStarted","Data":"0ab6814a4f34daa463a583c339723e91db30ef1f98e8e68d7ea380af738fae45"} Jul 07 14:28:26.658912 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:26.658878 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" event={"ID":"b13d5133-cbd7-4d28-8f88-ea22a4a14014","Type":"ContainerStarted","Data":"5c8fc7611b18ba2ac7beb88a176c6a0c62032f39d4703bb9584b27a4f881c452"} Jul 07 14:28:26.660533 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:26.660503 2555 generic.go:356] "Generic (PLEG): container finished" podID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerID="4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f" exitCode=0 Jul 07 14:28:26.660648 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:26.660589 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e300535-55b4-4e26-ab19-336e55d21a7e","Type":"ContainerDied","Data":"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f"} Jul 07 14:28:26.660713 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:26.660654 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e300535-55b4-4e26-ab19-336e55d21a7e","Type":"ContainerStarted","Data":"bbeaa3c3b37c4b0dba86ec41da4b13c7ad7964af82451ec193791c472363c0d7"} Jul 07 14:28:26.662412 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:26.662389 2555 generic.go:356] "Generic (PLEG): container finished" podID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerID="4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e" exitCode=0 Jul 07 14:28:26.662588 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:26.662444 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02","Type":"ContainerDied","Data":"4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e"} Jul 07 14:28:26.664489 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:26.664467 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-bbblk" Jul 07 14:28:26.673060 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:26.673012 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-bbblk" podStartSLOduration=1.240674405 podStartE2EDuration="21.672995705s" totalImagesPullingTime="20.4323213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:28:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:26.672141238 +0000 UTC m=+108.020194792" watchObservedRunningTime="2026-07-07 14:28:26.672995705 +0000 UTC m=+108.021049259" Jul 07 14:28:27.106603 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:27.106562 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rzvgc_fffeadca-9677-487f-a937-65985edbba4d/init-textfile/0.log" Jul 07 14:28:27.308618 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:27.308543 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rzvgc_fffeadca-9677-487f-a937-65985edbba4d/node-exporter/0.log" Jul 07 14:28:27.514366 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:27.514271 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rzvgc_fffeadca-9677-487f-a937-65985edbba4d/kube-rbac-proxy/0.log" Jul 07 14:28:27.674521 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:27.674482 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" event={"ID":"b13d5133-cbd7-4d28-8f88-ea22a4a14014","Type":"ContainerStarted","Data":"7985f9dae3b2592135b03ac5c66c09012a7b80ed131edd308029bba67b85b4e7"} Jul 07 14:28:27.674705 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:27.674559 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" event={"ID":"b13d5133-cbd7-4d28-8f88-ea22a4a14014","Type":"ContainerStarted","Data":"7a974795732c959d8f7dd72ef5e00d00a3c63d2ff54db51e211f4bb18258adc3"} Jul 07 14:28:27.674705 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:27.674601 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" event={"ID":"b13d5133-cbd7-4d28-8f88-ea22a4a14014","Type":"ContainerStarted","Data":"d7a22a63b3ec02658fe0e1bc207eea2da932d38d0683f17c988d5a0bbfc30ab0"} Jul 07 14:28:27.674992 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:27.674973 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:27.698097 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:27.696787 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" podStartSLOduration=2.020440197 podStartE2EDuration="11.696767436s" totalImagesPullingTime="9.676327239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:28:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:27.695720211 +0000 UTC m=+109.043773767" watchObservedRunningTime="2026-07-07 14:28:27.696767436 +0000 UTC m=+109.044820991" Jul 07 14:28:28.306941 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:28.306901 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e300535-55b4-4e26-ab19-336e55d21a7e/init-config-reloader/0.log" Jul 07 14:28:30.310783 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:30.310704 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65fdbf6b89-sfgth_b13d5133-cbd7-4d28-8f88-ea22a4a14014/thanos-query/0.log" Jul 07 14:28:30.506928 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:30.506771 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65fdbf6b89-sfgth_b13d5133-cbd7-4d28-8f88-ea22a4a14014/kube-rbac-proxy-web/0.log" Jul 07 14:28:30.706170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:30.706091 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65fdbf6b89-sfgth_b13d5133-cbd7-4d28-8f88-ea22a4a14014/kube-rbac-proxy/0.log" Jul 07 14:28:30.907387 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:30.907360 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65fdbf6b89-sfgth_b13d5133-cbd7-4d28-8f88-ea22a4a14014/prom-label-proxy/0.log" Jul 07 14:28:31.106641 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:31.106613 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65fdbf6b89-sfgth_b13d5133-cbd7-4d28-8f88-ea22a4a14014/kube-rbac-proxy-rules/0.log" Jul 07 14:28:31.306356 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:31.306330 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65fdbf6b89-sfgth_b13d5133-cbd7-4d28-8f88-ea22a4a14014/kube-rbac-proxy-metrics/0.log" Jul 07 14:28:31.698888 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:31.698804 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02","Type":"ContainerStarted","Data":"bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c"} Jul 07 14:28:31.699336 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:31.698916 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02","Type":"ContainerStarted","Data":"3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e"} Jul 07 14:28:31.699336 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:31.698962 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02","Type":"ContainerStarted","Data":"3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b"} Jul 07 14:28:31.699336 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:31.699000 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02","Type":"ContainerStarted","Data":"e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f"} Jul 07 14:28:31.699336 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:31.699041 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02","Type":"ContainerStarted","Data":"9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35"} Jul 07 14:28:31.711089 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:31.703484 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e300535-55b4-4e26-ab19-336e55d21a7e","Type":"ContainerStarted","Data":"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05"} Jul 07 14:28:31.711089 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:31.703555 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e300535-55b4-4e26-ab19-336e55d21a7e","Type":"ContainerStarted","Data":"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d"} Jul 07 14:28:31.711089 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:31.703593 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e300535-55b4-4e26-ab19-336e55d21a7e","Type":"ContainerStarted","Data":"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2"} Jul 07 14:28:32.308480 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:32.308441 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-bbblk_b705da46-8252-44d5-ae93-2bb5f79b5b97/download-server/0.log" Jul 07 14:28:32.705668 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:32.705576 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7r4qc_88fc15e7-95c6-453e-afa7-e9b3ae0384ec/serve-healthcheck-canary/0.log" Jul 07 14:28:32.712445 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:32.712410 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e300535-55b4-4e26-ab19-336e55d21a7e","Type":"ContainerStarted","Data":"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd"} Jul 07 14:28:32.712610 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:32.712491 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e300535-55b4-4e26-ab19-336e55d21a7e","Type":"ContainerStarted","Data":"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644"} Jul 07 14:28:32.712610 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:32.712535 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e300535-55b4-4e26-ab19-336e55d21a7e","Type":"ContainerStarted","Data":"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832"} Jul 07 14:28:32.716446 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:32.716414 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02","Type":"ContainerStarted","Data":"ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1"} Jul 07 14:28:32.746782 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:32.746722 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=8.206643788 podStartE2EDuration="12.746702203s" totalImagesPullingTime="4.516071121s" totalInitContainerRuntime="23.987294ms" isStatefulPod=false podCreationTimestamp="2026-07-07 14:28:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:32.745570146 +0000 UTC m=+114.093623700" watchObservedRunningTime="2026-07-07 14:28:32.746702203 +0000 UTC m=+114.094755759" Jul 07 14:28:32.774827 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:32.774761 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.293890645 podStartE2EDuration="17.774740915s" totalImagesPullingTime="13.45567602s" totalInitContainerRuntime="25.17425ms" isStatefulPod=false podCreationTimestamp="2026-07-07 14:28:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:32.772761874 +0000 UTC m=+114.120815438" watchObservedRunningTime="2026-07-07 14:28:32.774740915 +0000 UTC m=+114.122794468" Jul 07 14:28:33.690490 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:33.690459 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-65fdbf6b89-sfgth" Jul 07 14:28:36.318437 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:28:36.318403 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:21.318607 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:21.318575 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:21.339190 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:21.339162 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:21.918832 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:21.918800 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:35.002321 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:35.002226 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 14:29:35.002854 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:35.002564 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="alertmanager" containerID="cri-o://9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35" gracePeriod=120 Jul 07 14:29:35.002854 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:35.002623 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="kube-rbac-proxy-web" containerID="cri-o://3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b" gracePeriod=120 Jul 07 14:29:35.002854 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:35.002655 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="prom-label-proxy" containerID="cri-o://ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1" gracePeriod=120 Jul 07 14:29:35.002854 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:35.002683 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="kube-rbac-proxy" containerID="cri-o://3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e" gracePeriod=120 Jul 07 14:29:35.002854 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:35.002619 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="kube-rbac-proxy-metric" containerID="cri-o://bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c" gracePeriod=120 Jul 07 14:29:35.002854 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:35.002679 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="config-reloader" containerID="cri-o://e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f" gracePeriod=120 Jul 07 14:29:35.958916 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:35.958884 2555 generic.go:356] "Generic (PLEG): container finished" podID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerID="ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1" exitCode=0 Jul 07 14:29:35.958916 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:35.958913 2555 generic.go:356] "Generic (PLEG): container finished" podID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerID="3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e" exitCode=0 Jul 07 14:29:35.958916 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:35.958921 2555 generic.go:356] "Generic (PLEG): container finished" podID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerID="e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f" exitCode=0 Jul 07 14:29:35.958916 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:35.958928 2555 generic.go:356] "Generic (PLEG): container finished" podID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerID="9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35" exitCode=0 Jul 07 14:29:35.959213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:35.958966 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02","Type":"ContainerDied","Data":"ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1"} Jul 07 14:29:35.959213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:35.959033 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02","Type":"ContainerDied","Data":"3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e"} Jul 07 14:29:35.959213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:35.959060 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02","Type":"ContainerDied","Data":"e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f"} Jul 07 14:29:35.959213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:35.959108 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02","Type":"ContainerDied","Data":"9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35"} Jul 07 14:29:36.367238 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.367206 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:36.482226 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482148 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-alertmanager-main-db\") pod \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " Jul 07 14:29:36.482226 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482183 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-cluster-tls-config\") pod \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " Jul 07 14:29:36.482226 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482224 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-alertmanager-trusted-ca-bundle\") pod \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " Jul 07 14:29:36.482508 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482255 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-config-out\") pod \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " Jul 07 14:29:36.482508 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482285 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-main-tls\") pod \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " Jul 07 14:29:36.482508 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482311 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy-web\") pod \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " Jul 07 14:29:36.482508 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482329 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75j46\" (UniqueName: \"kubernetes.io/projected/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-kube-api-access-75j46\") pod \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " Jul 07 14:29:36.482508 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482365 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-metrics-client-ca\") pod \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " Jul 07 14:29:36.482894 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482531 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" (UID: "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:29:36.482894 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482599 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-tls-assets\") pod \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " Jul 07 14:29:36.482894 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482605 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" (UID: "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:29:36.482894 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482632 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy\") pod \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " Jul 07 14:29:36.482894 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482666 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-web-config\") pod \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " Jul 07 14:29:36.482894 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482708 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " Jul 07 14:29:36.482894 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482742 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-config-volume\") pod \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\" (UID: \"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02\") " Jul 07 14:29:36.482894 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.482780 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" (UID: "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:29:36.483418 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.483017 2555 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-metrics-client-ca\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:36.483418 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.483326 2555 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-alertmanager-main-db\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:36.483418 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.483349 2555 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:36.485385 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.485329 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" (UID: "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:36.485856 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.485813 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-kube-api-access-75j46" (OuterVolumeSpecName: "kube-api-access-75j46") pod "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" (UID: "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02"). InnerVolumeSpecName "kube-api-access-75j46". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:29:36.486038 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.485996 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-config-out" (OuterVolumeSpecName: "config-out") pod "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" (UID: "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:29:36.486194 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.486170 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" (UID: "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:36.486379 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.486333 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" (UID: "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:36.486653 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.486630 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" (UID: "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:36.487537 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.487515 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" (UID: "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:29:36.487734 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.487715 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-config-volume" (OuterVolumeSpecName: "config-volume") pod "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" (UID: "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:36.489858 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.489764 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" (UID: "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:36.496552 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.496533 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-web-config" (OuterVolumeSpecName: "web-config") pod "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" (UID: "4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:36.584105 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.584039 2555 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:36.584105 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.584104 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75j46\" (UniqueName: \"kubernetes.io/projected/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-kube-api-access-75j46\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:36.584105 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.584115 2555 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-tls-assets\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:36.584319 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.584125 2555 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:36.584319 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.584133 2555 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-web-config\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:36.584319 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.584142 2555 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:36.584319 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.584152 2555 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-config-volume\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:36.584319 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.584160 2555 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-cluster-tls-config\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:36.584319 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.584168 2555 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-config-out\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:36.584319 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.584176 2555 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02-secret-alertmanager-main-tls\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:36.965994 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.965899 2555 generic.go:356] "Generic (PLEG): container finished" podID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerID="bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c" exitCode=0 Jul 07 14:29:36.965994 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.965932 2555 generic.go:356] "Generic (PLEG): container finished" podID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerID="3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b" exitCode=0 Jul 07 14:29:36.965994 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.965992 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:36.966262 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.965990 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02","Type":"ContainerDied","Data":"bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c"} Jul 07 14:29:36.966262 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.966103 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02","Type":"ContainerDied","Data":"3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b"} Jul 07 14:29:36.966262 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.966149 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02","Type":"ContainerDied","Data":"c66ad2e6a8da5acba908710896af024d7bcde5a5732391022627fb7e61f7d77c"} Jul 07 14:29:36.966262 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.966177 2555 scope.go:117] "RemoveContainer" containerID="ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1" Jul 07 14:29:36.975043 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.975025 2555 scope.go:117] "RemoveContainer" containerID="bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c" Jul 07 14:29:36.982047 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.982034 2555 scope.go:117] "RemoveContainer" containerID="3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e" Jul 07 14:29:36.987551 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.987530 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 14:29:36.989453 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.989436 2555 scope.go:117] "RemoveContainer" containerID="3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b" Jul 07 14:29:36.994518 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.994499 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 14:29:36.996764 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:36.996750 2555 scope.go:117] "RemoveContainer" containerID="e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f" Jul 07 14:29:37.003700 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.003682 2555 scope.go:117] "RemoveContainer" containerID="9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35" Jul 07 14:29:37.010427 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.010413 2555 scope.go:117] "RemoveContainer" containerID="4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e" Jul 07 14:29:37.017112 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.017081 2555 scope.go:117] "RemoveContainer" containerID="ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1" Jul 07 14:29:37.017400 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:29:37.017375 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1\": container with ID starting with ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1 not found: ID does not exist" containerID="ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1" Jul 07 14:29:37.017478 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.017407 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1"} err="failed to get container status \"ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1\": rpc error: code = NotFound desc = could not find container \"ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1\": container with ID starting with ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1 not found: ID does not exist" Jul 07 14:29:37.017478 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.017441 2555 scope.go:117] "RemoveContainer" containerID="bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c" Jul 07 14:29:37.017688 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:29:37.017672 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c\": container with ID starting with bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c not found: ID does not exist" containerID="bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c" Jul 07 14:29:37.017733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.017693 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c"} err="failed to get container status \"bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c\": rpc error: code = NotFound desc = could not find container \"bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c\": container with ID starting with bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c not found: ID does not exist" Jul 07 14:29:37.017733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.017708 2555 scope.go:117] "RemoveContainer" containerID="3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e" Jul 07 14:29:37.017949 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:29:37.017931 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e\": container with ID starting with 3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e not found: ID does not exist" containerID="3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e" Jul 07 14:29:37.017989 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.017954 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e"} err="failed to get container status \"3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e\": rpc error: code = NotFound desc = could not find container \"3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e\": container with ID starting with 3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e not found: ID does not exist" Jul 07 14:29:37.017989 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.017970 2555 scope.go:117] "RemoveContainer" containerID="3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b" Jul 07 14:29:37.018290 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:29:37.018273 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b\": container with ID starting with 3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b not found: ID does not exist" containerID="3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b" Jul 07 14:29:37.018352 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.018294 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b"} err="failed to get container status \"3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b\": rpc error: code = NotFound desc = could not find container \"3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b\": container with ID starting with 3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b not found: ID does not exist" Jul 07 14:29:37.018352 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.018308 2555 scope.go:117] "RemoveContainer" containerID="e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f" Jul 07 14:29:37.018567 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:29:37.018550 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f\": container with ID starting with e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f not found: ID does not exist" containerID="e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f" Jul 07 14:29:37.018603 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.018570 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f"} err="failed to get container status \"e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f\": rpc error: code = NotFound desc = could not find container \"e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f\": container with ID starting with e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f not found: ID does not exist" Jul 07 14:29:37.018603 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.018583 2555 scope.go:117] "RemoveContainer" containerID="9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35" Jul 07 14:29:37.018797 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:29:37.018781 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35\": container with ID starting with 9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35 not found: ID does not exist" containerID="9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35" Jul 07 14:29:37.018839 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.018801 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35"} err="failed to get container status \"9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35\": rpc error: code = NotFound desc = could not find container \"9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35\": container with ID starting with 9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35 not found: ID does not exist" Jul 07 14:29:37.018839 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.018814 2555 scope.go:117] "RemoveContainer" containerID="4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e" Jul 07 14:29:37.019113 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:29:37.019049 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e\": container with ID starting with 4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e not found: ID does not exist" containerID="4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e" Jul 07 14:29:37.019113 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.019090 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e"} err="failed to get container status \"4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e\": rpc error: code = NotFound desc = could not find container \"4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e\": container with ID starting with 4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e not found: ID does not exist" Jul 07 14:29:37.019113 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.019107 2555 scope.go:117] "RemoveContainer" containerID="ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1" Jul 07 14:29:37.019525 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.019301 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1"} err="failed to get container status \"ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1\": rpc error: code = NotFound desc = could not find container \"ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1\": container with ID starting with ebdd31b24aa3db37e6d762bfa5e0afc2a4dd7db41d107ed8d141d602c2e333a1 not found: ID does not exist" Jul 07 14:29:37.019525 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.019316 2555 scope.go:117] "RemoveContainer" containerID="bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c" Jul 07 14:29:37.019646 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.019605 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c"} err="failed to get container status \"bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c\": rpc error: code = NotFound desc = could not find container \"bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c\": container with ID starting with bcbb040f35f02116a8b565f291d56652a1a2b08b83671474059d8e504551839c not found: ID does not exist" Jul 07 14:29:37.019646 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.019626 2555 scope.go:117] "RemoveContainer" containerID="3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e" Jul 07 14:29:37.019923 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.019903 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e"} err="failed to get container status \"3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e\": rpc error: code = NotFound desc = could not find container \"3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e\": container with ID starting with 3702300a60552d2eac708d8b0fa997c66a8682fb93ad7c0ca9ec5d613209ee0e not found: ID does not exist" Jul 07 14:29:37.019923 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.019923 2555 scope.go:117] "RemoveContainer" containerID="3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b" Jul 07 14:29:37.020194 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.020167 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b"} err="failed to get container status \"3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b\": rpc error: code = NotFound desc = could not find container \"3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b\": container with ID starting with 3c38713eea191556757ce27ea75aaefa57c7f91c43c252b7378fdf308038b39b not found: ID does not exist" Jul 07 14:29:37.020194 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.020193 2555 scope.go:117] "RemoveContainer" containerID="e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f" Jul 07 14:29:37.020446 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.020429 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f"} err="failed to get container status \"e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f\": rpc error: code = NotFound desc = could not find container \"e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f\": container with ID starting with e4ff1313ee8cec396f32a6429f9134c1bcef82fdb23c76a70ca51ae5590aa33f not found: ID does not exist" Jul 07 14:29:37.020502 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.020447 2555 scope.go:117] "RemoveContainer" containerID="9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35" Jul 07 14:29:37.020669 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.020649 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35"} err="failed to get container status \"9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35\": rpc error: code = NotFound desc = could not find container \"9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35\": container with ID starting with 9efd3ad3bca0e1a6a9d00df06436ece92daa3d3e72e66de40a1d575800dbdd35 not found: ID does not exist" Jul 07 14:29:37.020669 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.020668 2555 scope.go:117] "RemoveContainer" containerID="4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e" Jul 07 14:29:37.020906 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.020886 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e"} err="failed to get container status \"4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e\": rpc error: code = NotFound desc = could not find container \"4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e\": container with ID starting with 4e8d345f6bc9efb40df895f0858b95795d2b71b71ae05fc06e2299ee7f9cd54e not found: ID does not exist" Jul 07 14:29:37.028554 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028535 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 14:29:37.028796 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028785 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="kube-rbac-proxy-metric" Jul 07 14:29:37.028838 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028798 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="kube-rbac-proxy-metric" Jul 07 14:29:37.028838 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028810 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="config-reloader" Jul 07 14:29:37.028838 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028816 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="config-reloader" Jul 07 14:29:37.028838 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028827 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="kube-rbac-proxy-web" Jul 07 14:29:37.028838 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028832 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="kube-rbac-proxy-web" Jul 07 14:29:37.028981 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028842 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="prom-label-proxy" Jul 07 14:29:37.028981 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028847 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="prom-label-proxy" Jul 07 14:29:37.028981 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028855 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="init-config-reloader" Jul 07 14:29:37.028981 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028861 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="init-config-reloader" Jul 07 14:29:37.028981 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028866 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="alertmanager" Jul 07 14:29:37.028981 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028871 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="alertmanager" Jul 07 14:29:37.028981 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028878 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="kube-rbac-proxy" Jul 07 14:29:37.028981 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028884 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="kube-rbac-proxy" Jul 07 14:29:37.028981 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028928 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="kube-rbac-proxy-metric" Jul 07 14:29:37.028981 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028936 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="kube-rbac-proxy-web" Jul 07 14:29:37.028981 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028943 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="kube-rbac-proxy" Jul 07 14:29:37.028981 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028949 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="alertmanager" Jul 07 14:29:37.028981 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028954 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="config-reloader" Jul 07 14:29:37.028981 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.028959 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" containerName="prom-label-proxy" Jul 07 14:29:37.032637 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.032620 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.034936 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.034917 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 14:29:37.035089 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.035051 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 14:29:37.035317 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.035298 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 14:29:37.035317 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.035313 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 14:29:37.035447 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.035301 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 14:29:37.035580 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.035564 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 14:29:37.035667 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.035566 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 14:29:37.035667 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.035622 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 14:29:37.036271 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.036251 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-k2q8h\"" Jul 07 14:29:37.043473 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.043455 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 14:29:37.053883 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.053855 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 14:29:37.088843 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.088821 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwnnt\" (UniqueName: \"kubernetes.io/projected/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-kube-api-access-xwnnt\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.088955 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.088855 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.088955 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.088876 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.088955 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.088903 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.088955 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.088935 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-config-out\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.089152 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.088957 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-web-config\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.089152 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.088982 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.089152 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.089006 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.089152 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.089036 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.089152 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.089056 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.089152 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.089110 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.089332 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.089151 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-config-volume\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.089332 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.089195 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.178837 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.178795 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02" path="/var/lib/kubelet/pods/4e3bdde3-1587-4ca4-bc7a-2e0718ae0d02/volumes" Jul 07 14:29:37.190243 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.190217 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.190243 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.190244 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-config-out\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.190416 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.190265 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-web-config\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.190416 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.190284 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.190416 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.190302 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.190416 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.190339 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.190416 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.190362 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.190416 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.190383 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.190704 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.190422 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-config-volume\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.190704 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.190446 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.190704 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.190472 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwnnt\" (UniqueName: \"kubernetes.io/projected/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-kube-api-access-xwnnt\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.190704 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.190497 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.190704 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.190524 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.191716 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.191393 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.192030 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.192006 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.192146 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.192096 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.193684 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.193652 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-web-config\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.193781 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.193704 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.193860 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.193836 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-config-out\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.193947 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.193887 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.194653 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.194624 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.194725 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.194711 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.194821 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.194802 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-config-volume\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.194977 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.194959 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.195029 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.195015 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.199636 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.199615 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwnnt\" (UniqueName: \"kubernetes.io/projected/e28fbf92-8bc8-44c1-84f2-e90cb01d91f3-kube-api-access-xwnnt\") pod \"alertmanager-main-0\" (UID: \"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.342790 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.342755 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 14:29:37.485719 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.485697 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 14:29:37.488340 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:29:37.488310 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28fbf92_8bc8_44c1_84f2_e90cb01d91f3.slice/crio-dc8215c4c47cee518dc295e0f54a10c4d105ef8f3e56c57fb355d7b97a032211 WatchSource:0}: Error finding container dc8215c4c47cee518dc295e0f54a10c4d105ef8f3e56c57fb355d7b97a032211: Status 404 returned error can't find the container with id dc8215c4c47cee518dc295e0f54a10c4d105ef8f3e56c57fb355d7b97a032211 Jul 07 14:29:37.576863 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:29:37.576838 2555 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28fbf92_8bc8_44c1_84f2_e90cb01d91f3.slice/crio-c03d287624f64c7e4e4c5cdfc03e135dba17dbb2319df9978a44d0d262dcc943.scope\": RecentStats: unable to find data in memory cache]" Jul 07 14:29:37.971333 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.971297 2555 generic.go:356] "Generic (PLEG): container finished" podID="e28fbf92-8bc8-44c1-84f2-e90cb01d91f3" containerID="c03d287624f64c7e4e4c5cdfc03e135dba17dbb2319df9978a44d0d262dcc943" exitCode=0 Jul 07 14:29:37.971517 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.971351 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3","Type":"ContainerDied","Data":"c03d287624f64c7e4e4c5cdfc03e135dba17dbb2319df9978a44d0d262dcc943"} Jul 07 14:29:37.971517 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:37.971415 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3","Type":"ContainerStarted","Data":"dc8215c4c47cee518dc295e0f54a10c4d105ef8f3e56c57fb355d7b97a032211"} Jul 07 14:29:38.979928 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:38.979891 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3","Type":"ContainerStarted","Data":"2a0d393c377ae404fe9f72df4105f172cbcfd218b31b8aef6cbbd8b0cec1de0a"} Jul 07 14:29:38.980356 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:38.979971 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3","Type":"ContainerStarted","Data":"e44017d3c8c04a4ee4c377ffdfd635f6614435a72d8b1cd3ec816418e851f3ec"} Jul 07 14:29:38.980356 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:38.980012 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3","Type":"ContainerStarted","Data":"d4efe6b73b5d94057cddd6f4dac60839121ef4ad96ac3bf54b60540ff9567ed7"} Jul 07 14:29:38.980356 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:38.980055 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3","Type":"ContainerStarted","Data":"0b40c9cb657588008ac17b846da73a09e5a844817e29ca3e2792109768fc07ac"} Jul 07 14:29:38.980356 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:38.980115 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3","Type":"ContainerStarted","Data":"1b62e3b841892766e3ca4743373ba90de1682a6f689b955f823ab4809c844c15"} Jul 07 14:29:38.980356 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:38.980157 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e28fbf92-8bc8-44c1-84f2-e90cb01d91f3","Type":"ContainerStarted","Data":"3202585ea54a744c49ca90d10d0186715d66eb23c3f2ac8f187fdeb41417ff6e"} Jul 07 14:29:39.013428 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.013381 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.994489602 podStartE2EDuration="2.013365738s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.876136ms" isStatefulPod=false podCreationTimestamp="2026-07-07 14:29:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:29:39.012282833 +0000 UTC m=+180.360336379" watchObservedRunningTime="2026-07-07 14:29:39.013365738 +0000 UTC m=+180.361419291" Jul 07 14:29:39.340883 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.340796 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 14:29:39.341146 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.341115 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="prometheus" containerID="cri-o://747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2" gracePeriod=600 Jul 07 14:29:39.341242 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.341151 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="thanos-sidecar" containerID="cri-o://1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05" gracePeriod=600 Jul 07 14:29:39.341242 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.341190 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="config-reloader" containerID="cri-o://6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d" gracePeriod=600 Jul 07 14:29:39.341354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.341151 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="kube-rbac-proxy-web" containerID="cri-o://92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832" gracePeriod=600 Jul 07 14:29:39.341354 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.341186 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="kube-rbac-proxy" containerID="cri-o://6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644" gracePeriod=600 Jul 07 14:29:39.341453 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.341403 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="kube-rbac-proxy-thanos" containerID="cri-o://863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd" gracePeriod=600 Jul 07 14:29:39.574845 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.574822 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:39.712093 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.711987 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-trusted-ca-bundle\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712093 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712020 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712093 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712040 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-kubelet-serving-ca-bundle\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712093 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712085 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-config\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712423 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712120 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e300535-55b4-4e26-ab19-336e55d21a7e-tls-assets\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712423 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712212 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-kube-rbac-proxy\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712423 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712250 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-thanos-prometheus-http-client-file\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712423 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712279 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-metrics-client-ca\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712423 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712307 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-2\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712423 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712344 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-serving-certs-ca-bundle\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712423 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712386 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-1\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712423 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712410 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-0\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712779 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712437 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712779 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712468 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e300535-55b4-4e26-ab19-336e55d21a7e-config-out\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712779 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712461 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:29:39.712779 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712493 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-grpc-tls\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712779 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712514 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:29:39.712779 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712543 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-web-config\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712779 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712578 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-db\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712779 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712611 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-tls\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712779 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712640 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbd86\" (UniqueName: \"kubernetes.io/projected/3e300535-55b4-4e26-ab19-336e55d21a7e-kube-api-access-tbd86\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.712779 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712667 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-metrics-client-certs\") pod \"3e300535-55b4-4e26-ab19-336e55d21a7e\" (UID: \"3e300535-55b4-4e26-ab19-336e55d21a7e\") " Jul 07 14:29:39.713277 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712874 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:29:39.713277 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.712894 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:29:39.713277 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.713253 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:29:39.713549 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.713525 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:29:39.714638 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.714160 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:29:39.715146 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.715100 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:29:39.715400 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.715379 2555 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-db\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.715479 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.715406 2555 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.715479 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.715422 2555 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.715479 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.715436 2555 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-metrics-client-ca\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.715479 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.715450 2555 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.715479 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.715462 2555 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.715479 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.715474 2555 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.715756 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.715486 2555 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3e300535-55b4-4e26-ab19-336e55d21a7e-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.715808 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.715744 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:39.715808 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.715771 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-config" (OuterVolumeSpecName: "config") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:39.715808 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.715774 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e300535-55b4-4e26-ab19-336e55d21a7e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:29:39.715808 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.715798 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:39.716134 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.716050 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:39.716250 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.716186 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:39.716799 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.716780 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:39.717563 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.717533 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:39.717629 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.717596 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:39.718031 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.718010 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e300535-55b4-4e26-ab19-336e55d21a7e-config-out" (OuterVolumeSpecName: "config-out") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:29:39.718181 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.718164 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e300535-55b4-4e26-ab19-336e55d21a7e-kube-api-access-tbd86" (OuterVolumeSpecName: "kube-api-access-tbd86") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "kube-api-access-tbd86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:29:39.727450 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.727433 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-web-config" (OuterVolumeSpecName: "web-config") pod "3e300535-55b4-4e26-ab19-336e55d21a7e" (UID: "3e300535-55b4-4e26-ab19-336e55d21a7e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 14:29:39.815971 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.815940 2555 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.815971 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.815966 2555 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e300535-55b4-4e26-ab19-336e55d21a7e-config-out\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.815971 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.815975 2555 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-grpc-tls\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.816208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.816004 2555 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-web-config\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.816208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.816013 2555 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.816208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.816023 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbd86\" (UniqueName: \"kubernetes.io/projected/3e300535-55b4-4e26-ab19-336e55d21a7e-kube-api-access-tbd86\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.816208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.816031 2555 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-metrics-client-certs\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.816208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.816039 2555 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.816208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.816047 2555 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-config\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.816208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.816055 2555 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e300535-55b4-4e26-ab19-336e55d21a7e-tls-assets\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.816208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.816093 2555 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-secret-kube-rbac-proxy\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.816208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.816106 2555 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3e300535-55b4-4e26-ab19-336e55d21a7e-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:29:39.987106 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.987057 2555 generic.go:356] "Generic (PLEG): container finished" podID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerID="863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd" exitCode=0 Jul 07 14:29:39.987106 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.987097 2555 generic.go:356] "Generic (PLEG): container finished" podID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerID="6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644" exitCode=0 Jul 07 14:29:39.987106 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.987103 2555 generic.go:356] "Generic (PLEG): container finished" podID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerID="92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832" exitCode=0 Jul 07 14:29:39.987106 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.987109 2555 generic.go:356] "Generic (PLEG): container finished" podID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerID="1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05" exitCode=0 Jul 07 14:29:39.987106 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.987114 2555 generic.go:356] "Generic (PLEG): container finished" podID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerID="6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d" exitCode=0 Jul 07 14:29:39.987636 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.987119 2555 generic.go:356] "Generic (PLEG): container finished" podID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerID="747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2" exitCode=0 Jul 07 14:29:39.987636 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.987138 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e300535-55b4-4e26-ab19-336e55d21a7e","Type":"ContainerDied","Data":"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd"} Jul 07 14:29:39.987636 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.987210 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e300535-55b4-4e26-ab19-336e55d21a7e","Type":"ContainerDied","Data":"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644"} Jul 07 14:29:39.987636 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.987146 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:39.987636 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.987242 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e300535-55b4-4e26-ab19-336e55d21a7e","Type":"ContainerDied","Data":"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832"} Jul 07 14:29:39.987636 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.987270 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e300535-55b4-4e26-ab19-336e55d21a7e","Type":"ContainerDied","Data":"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05"} Jul 07 14:29:39.987636 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.987295 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e300535-55b4-4e26-ab19-336e55d21a7e","Type":"ContainerDied","Data":"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d"} Jul 07 14:29:39.987636 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.987312 2555 scope.go:117] "RemoveContainer" containerID="863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd" Jul 07 14:29:39.987636 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.987318 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e300535-55b4-4e26-ab19-336e55d21a7e","Type":"ContainerDied","Data":"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2"} Jul 07 14:29:39.987636 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.987428 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e300535-55b4-4e26-ab19-336e55d21a7e","Type":"ContainerDied","Data":"bbeaa3c3b37c4b0dba86ec41da4b13c7ad7964af82451ec193791c472363c0d7"} Jul 07 14:29:39.995742 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:39.995725 2555 scope.go:117] "RemoveContainer" containerID="6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644" Jul 07 14:29:40.002556 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.002541 2555 scope.go:117] "RemoveContainer" containerID="92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832" Jul 07 14:29:40.009133 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.009114 2555 scope.go:117] "RemoveContainer" containerID="1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05" Jul 07 14:29:40.010766 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.010749 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 14:29:40.015241 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.015221 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 14:29:40.016381 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.016266 2555 scope.go:117] "RemoveContainer" containerID="6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d" Jul 07 14:29:40.022874 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.022857 2555 scope.go:117] "RemoveContainer" containerID="747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2" Jul 07 14:29:40.029721 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.029708 2555 scope.go:117] "RemoveContainer" containerID="4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f" Jul 07 14:29:40.036324 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.036310 2555 scope.go:117] "RemoveContainer" containerID="863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd" Jul 07 14:29:40.036574 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:29:40.036557 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd\": container with ID starting with 863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd not found: ID does not exist" containerID="863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd" Jul 07 14:29:40.036625 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.036583 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd"} err="failed to get container status \"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd\": rpc error: code = NotFound desc = could not find container \"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd\": container with ID starting with 863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd not found: ID does not exist" Jul 07 14:29:40.036625 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.036600 2555 scope.go:117] "RemoveContainer" containerID="6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644" Jul 07 14:29:40.036832 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:29:40.036814 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644\": container with ID starting with 6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644 not found: ID does not exist" containerID="6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644" Jul 07 14:29:40.036893 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.036840 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644"} err="failed to get container status \"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644\": rpc error: code = NotFound desc = could not find container \"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644\": container with ID starting with 6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644 not found: ID does not exist" Jul 07 14:29:40.036893 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.036863 2555 scope.go:117] "RemoveContainer" containerID="92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832" Jul 07 14:29:40.037141 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:29:40.037120 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832\": container with ID starting with 92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832 not found: ID does not exist" containerID="92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832" Jul 07 14:29:40.037184 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.037145 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832"} err="failed to get container status \"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832\": rpc error: code = NotFound desc = could not find container \"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832\": container with ID starting with 92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832 not found: ID does not exist" Jul 07 14:29:40.037184 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.037161 2555 scope.go:117] "RemoveContainer" containerID="1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05" Jul 07 14:29:40.037396 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:29:40.037381 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05\": container with ID starting with 1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05 not found: ID does not exist" containerID="1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05" Jul 07 14:29:40.037441 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.037402 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05"} err="failed to get container status \"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05\": rpc error: code = NotFound desc = could not find container \"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05\": container with ID starting with 1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05 not found: ID does not exist" Jul 07 14:29:40.037441 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.037419 2555 scope.go:117] "RemoveContainer" containerID="6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d" Jul 07 14:29:40.037655 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:29:40.037637 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d\": container with ID starting with 6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d not found: ID does not exist" containerID="6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d" Jul 07 14:29:40.037716 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.037662 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d"} err="failed to get container status \"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d\": rpc error: code = NotFound desc = could not find container \"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d\": container with ID starting with 6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d not found: ID does not exist" Jul 07 14:29:40.037716 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.037683 2555 scope.go:117] "RemoveContainer" containerID="747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2" Jul 07 14:29:40.037936 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:29:40.037919 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2\": container with ID starting with 747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2 not found: ID does not exist" containerID="747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2" Jul 07 14:29:40.037982 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.037941 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2"} err="failed to get container status \"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2\": rpc error: code = NotFound desc = could not find container \"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2\": container with ID starting with 747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2 not found: ID does not exist" Jul 07 14:29:40.037982 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.037958 2555 scope.go:117] "RemoveContainer" containerID="4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f" Jul 07 14:29:40.038232 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:29:40.038212 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f\": container with ID starting with 4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f not found: ID does not exist" containerID="4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f" Jul 07 14:29:40.038274 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.038239 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f"} err="failed to get container status \"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f\": rpc error: code = NotFound desc = could not find container \"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f\": container with ID starting with 4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f not found: ID does not exist" Jul 07 14:29:40.038274 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.038258 2555 scope.go:117] "RemoveContainer" containerID="863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd" Jul 07 14:29:40.038489 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.038473 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd"} err="failed to get container status \"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd\": rpc error: code = NotFound desc = could not find container \"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd\": container with ID starting with 863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd not found: ID does not exist" Jul 07 14:29:40.038533 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.038489 2555 scope.go:117] "RemoveContainer" containerID="6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644" Jul 07 14:29:40.038703 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.038689 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644"} err="failed to get container status \"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644\": rpc error: code = NotFound desc = could not find container \"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644\": container with ID starting with 6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644 not found: ID does not exist" Jul 07 14:29:40.038743 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.038703 2555 scope.go:117] "RemoveContainer" containerID="92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832" Jul 07 14:29:40.038900 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.038882 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832"} err="failed to get container status \"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832\": rpc error: code = NotFound desc = could not find container \"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832\": container with ID starting with 92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832 not found: ID does not exist" Jul 07 14:29:40.038944 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.038901 2555 scope.go:117] "RemoveContainer" containerID="1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05" Jul 07 14:29:40.039121 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.039105 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05"} err="failed to get container status \"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05\": rpc error: code = NotFound desc = could not find container \"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05\": container with ID starting with 1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05 not found: ID does not exist" Jul 07 14:29:40.039168 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.039122 2555 scope.go:117] "RemoveContainer" containerID="6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d" Jul 07 14:29:40.039337 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.039317 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d"} err="failed to get container status \"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d\": rpc error: code = NotFound desc = could not find container \"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d\": container with ID starting with 6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d not found: ID does not exist" Jul 07 14:29:40.039373 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.039339 2555 scope.go:117] "RemoveContainer" containerID="747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2" Jul 07 14:29:40.039551 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.039536 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2"} err="failed to get container status \"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2\": rpc error: code = NotFound desc = could not find container \"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2\": container with ID starting with 747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2 not found: ID does not exist" Jul 07 14:29:40.039592 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.039551 2555 scope.go:117] "RemoveContainer" containerID="4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f" Jul 07 14:29:40.039765 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.039749 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f"} err="failed to get container status \"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f\": rpc error: code = NotFound desc = could not find container \"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f\": container with ID starting with 4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f not found: ID does not exist" Jul 07 14:29:40.039765 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.039764 2555 scope.go:117] "RemoveContainer" containerID="863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd" Jul 07 14:29:40.039959 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.039945 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd"} err="failed to get container status \"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd\": rpc error: code = NotFound desc = could not find container \"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd\": container with ID starting with 863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd not found: ID does not exist" Jul 07 14:29:40.039995 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.039959 2555 scope.go:117] "RemoveContainer" containerID="6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644" Jul 07 14:29:40.040177 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.040159 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644"} err="failed to get container status \"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644\": rpc error: code = NotFound desc = could not find container \"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644\": container with ID starting with 6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644 not found: ID does not exist" Jul 07 14:29:40.040225 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.040178 2555 scope.go:117] "RemoveContainer" containerID="92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832" Jul 07 14:29:40.040376 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.040360 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832"} err="failed to get container status \"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832\": rpc error: code = NotFound desc = could not find container \"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832\": container with ID starting with 92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832 not found: ID does not exist" Jul 07 14:29:40.040410 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.040376 2555 scope.go:117] "RemoveContainer" containerID="1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05" Jul 07 14:29:40.040562 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.040548 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05"} err="failed to get container status \"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05\": rpc error: code = NotFound desc = could not find container \"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05\": container with ID starting with 1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05 not found: ID does not exist" Jul 07 14:29:40.040603 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.040562 2555 scope.go:117] "RemoveContainer" containerID="6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d" Jul 07 14:29:40.040753 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.040731 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d"} err="failed to get container status \"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d\": rpc error: code = NotFound desc = could not find container \"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d\": container with ID starting with 6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d not found: ID does not exist" Jul 07 14:29:40.040806 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.040755 2555 scope.go:117] "RemoveContainer" containerID="747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2" Jul 07 14:29:40.040957 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.040934 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2"} err="failed to get container status \"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2\": rpc error: code = NotFound desc = could not find container \"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2\": container with ID starting with 747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2 not found: ID does not exist" Jul 07 14:29:40.040957 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.040956 2555 scope.go:117] "RemoveContainer" containerID="4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f" Jul 07 14:29:40.041280 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.041246 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f"} err="failed to get container status \"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f\": rpc error: code = NotFound desc = could not find container \"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f\": container with ID starting with 4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f not found: ID does not exist" Jul 07 14:29:40.041280 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.041271 2555 scope.go:117] "RemoveContainer" containerID="863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd" Jul 07 14:29:40.041602 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.041575 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd"} err="failed to get container status \"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd\": rpc error: code = NotFound desc = could not find container \"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd\": container with ID starting with 863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd not found: ID does not exist" Jul 07 14:29:40.041685 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.041641 2555 scope.go:117] "RemoveContainer" containerID="6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644" Jul 07 14:29:40.041928 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.041905 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644"} err="failed to get container status \"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644\": rpc error: code = NotFound desc = could not find container \"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644\": container with ID starting with 6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644 not found: ID does not exist" Jul 07 14:29:40.041991 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.041936 2555 scope.go:117] "RemoveContainer" containerID="92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832" Jul 07 14:29:40.042198 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.042182 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832"} err="failed to get container status \"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832\": rpc error: code = NotFound desc = could not find container \"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832\": container with ID starting with 92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832 not found: ID does not exist" Jul 07 14:29:40.042198 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.042198 2555 scope.go:117] "RemoveContainer" containerID="1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05" Jul 07 14:29:40.042427 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.042408 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05"} err="failed to get container status \"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05\": rpc error: code = NotFound desc = could not find container \"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05\": container with ID starting with 1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05 not found: ID does not exist" Jul 07 14:29:40.042472 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.042429 2555 scope.go:117] "RemoveContainer" containerID="6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d" Jul 07 14:29:40.042629 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.042609 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d"} err="failed to get container status \"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d\": rpc error: code = NotFound desc = could not find container \"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d\": container with ID starting with 6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d not found: ID does not exist" Jul 07 14:29:40.042629 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.042628 2555 scope.go:117] "RemoveContainer" containerID="747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2" Jul 07 14:29:40.042859 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.042843 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2"} err="failed to get container status \"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2\": rpc error: code = NotFound desc = could not find container \"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2\": container with ID starting with 747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2 not found: ID does not exist" Jul 07 14:29:40.042859 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.042852 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 14:29:40.042859 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.042860 2555 scope.go:117] "RemoveContainer" containerID="4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f" Jul 07 14:29:40.043052 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043037 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f"} err="failed to get container status \"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f\": rpc error: code = NotFound desc = could not find container \"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f\": container with ID starting with 4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f not found: ID does not exist" Jul 07 14:29:40.043107 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043053 2555 scope.go:117] "RemoveContainer" containerID="863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd" Jul 07 14:29:40.043201 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043184 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="init-config-reloader" Jul 07 14:29:40.043234 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043206 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="init-config-reloader" Jul 07 14:29:40.043234 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043224 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="thanos-sidecar" Jul 07 14:29:40.043302 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043234 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="thanos-sidecar" Jul 07 14:29:40.043302 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043244 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="kube-rbac-proxy" Jul 07 14:29:40.043302 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043252 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="kube-rbac-proxy" Jul 07 14:29:40.043302 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043250 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd"} err="failed to get container status \"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd\": rpc error: code = NotFound desc = could not find container \"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd\": container with ID starting with 863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd not found: ID does not exist" Jul 07 14:29:40.043302 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043258 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="kube-rbac-proxy-thanos" Jul 07 14:29:40.043302 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043264 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="kube-rbac-proxy-thanos" Jul 07 14:29:40.043302 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043278 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="prometheus" Jul 07 14:29:40.043302 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043283 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="prometheus" Jul 07 14:29:40.043302 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043297 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="kube-rbac-proxy-web" Jul 07 14:29:40.043302 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043302 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="kube-rbac-proxy-web" Jul 07 14:29:40.043652 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043310 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="config-reloader" Jul 07 14:29:40.043652 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043315 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="config-reloader" Jul 07 14:29:40.043652 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043264 2555 scope.go:117] "RemoveContainer" containerID="6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644" Jul 07 14:29:40.043652 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043355 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="thanos-sidecar" Jul 07 14:29:40.043652 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043364 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="prometheus" Jul 07 14:29:40.043652 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043370 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="kube-rbac-proxy" Jul 07 14:29:40.043652 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043376 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="kube-rbac-proxy-web" Jul 07 14:29:40.043652 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043382 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="kube-rbac-proxy-thanos" Jul 07 14:29:40.043652 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043389 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" containerName="config-reloader" Jul 07 14:29:40.043652 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043535 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644"} err="failed to get container status \"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644\": rpc error: code = NotFound desc = could not find container \"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644\": container with ID starting with 6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644 not found: ID does not exist" Jul 07 14:29:40.043652 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043557 2555 scope.go:117] "RemoveContainer" containerID="92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832" Jul 07 14:29:40.043994 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043754 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832"} err="failed to get container status \"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832\": rpc error: code = NotFound desc = could not find container \"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832\": container with ID starting with 92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832 not found: ID does not exist" Jul 07 14:29:40.043994 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043776 2555 scope.go:117] "RemoveContainer" containerID="1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05" Jul 07 14:29:40.043994 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043980 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05"} err="failed to get container status \"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05\": rpc error: code = NotFound desc = could not find container \"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05\": container with ID starting with 1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05 not found: ID does not exist" Jul 07 14:29:40.044125 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.043995 2555 scope.go:117] "RemoveContainer" containerID="6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d" Jul 07 14:29:40.044228 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.044209 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d"} err="failed to get container status \"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d\": rpc error: code = NotFound desc = could not find container \"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d\": container with ID starting with 6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d not found: ID does not exist" Jul 07 14:29:40.044291 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.044230 2555 scope.go:117] "RemoveContainer" containerID="747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2" Jul 07 14:29:40.044496 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.044477 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2"} err="failed to get container status \"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2\": rpc error: code = NotFound desc = could not find container \"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2\": container with ID starting with 747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2 not found: ID does not exist" Jul 07 14:29:40.044561 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.044496 2555 scope.go:117] "RemoveContainer" containerID="4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f" Jul 07 14:29:40.044721 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.044702 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f"} err="failed to get container status \"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f\": rpc error: code = NotFound desc = could not find container \"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f\": container with ID starting with 4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f not found: ID does not exist" Jul 07 14:29:40.044786 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.044722 2555 scope.go:117] "RemoveContainer" containerID="863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd" Jul 07 14:29:40.044909 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.044893 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd"} err="failed to get container status \"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd\": rpc error: code = NotFound desc = could not find container \"863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd\": container with ID starting with 863b553ec3b3b9c06a3c9e5047c07b01b6e07c2f12d756acea9d18e3ae2c8dfd not found: ID does not exist" Jul 07 14:29:40.044973 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.044910 2555 scope.go:117] "RemoveContainer" containerID="6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644" Jul 07 14:29:40.045120 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.045103 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644"} err="failed to get container status \"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644\": rpc error: code = NotFound desc = could not find container \"6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644\": container with ID starting with 6f2580b2b1f46eaeb56a4c7af4241bfa013a9c49e47cb4ddd6fd6bfe9bd0a644 not found: ID does not exist" Jul 07 14:29:40.045184 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.045121 2555 scope.go:117] "RemoveContainer" containerID="92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832" Jul 07 14:29:40.045327 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.045311 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832"} err="failed to get container status \"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832\": rpc error: code = NotFound desc = could not find container \"92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832\": container with ID starting with 92d02913dddbf32b015328c7911baa1d4a977752e333ed2247098fbd69e82832 not found: ID does not exist" Jul 07 14:29:40.045387 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.045329 2555 scope.go:117] "RemoveContainer" containerID="1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05" Jul 07 14:29:40.045558 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.045539 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05"} err="failed to get container status \"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05\": rpc error: code = NotFound desc = could not find container \"1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05\": container with ID starting with 1768bf2595636e8dc78754525f231b56d44be7bef23c0494744348867870fa05 not found: ID does not exist" Jul 07 14:29:40.045596 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.045559 2555 scope.go:117] "RemoveContainer" containerID="6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d" Jul 07 14:29:40.045749 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.045733 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d"} err="failed to get container status \"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d\": rpc error: code = NotFound desc = could not find container \"6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d\": container with ID starting with 6fa030c798c7423d3feee1a12c21c02706829c9e159f7c5390e44e4978d6a33d not found: ID does not exist" Jul 07 14:29:40.045817 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.045751 2555 scope.go:117] "RemoveContainer" containerID="747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2" Jul 07 14:29:40.046008 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.045992 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2"} err="failed to get container status \"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2\": rpc error: code = NotFound desc = could not find container \"747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2\": container with ID starting with 747db83ee427f0ac6be05d46c55281178791a8f4f65b6c1689e3e4a9d64fbdc2 not found: ID does not exist" Jul 07 14:29:40.046053 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.046008 2555 scope.go:117] "RemoveContainer" containerID="4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f" Jul 07 14:29:40.046292 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.046275 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f"} err="failed to get container status \"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f\": rpc error: code = NotFound desc = could not find container \"4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f\": container with ID starting with 4577681b0f2fc989e71b0f5eb638b3837cdc6fabd8fa0c3aacef1302724b176f not found: ID does not exist" Jul 07 14:29:40.047470 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.047453 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.049697 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.049681 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 14:29:40.049826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.049707 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 14:29:40.050189 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.050174 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 14:29:40.050189 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.050187 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 14:29:40.050480 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.050444 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 14:29:40.050480 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.050470 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kjzlf\"" Jul 07 14:29:40.050948 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.050931 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 14:29:40.051167 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.051152 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 14:29:40.051264 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.051246 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 14:29:40.051362 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.051320 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 14:29:40.051444 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.051371 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7vvq35ka0u0ds\"" Jul 07 14:29:40.051564 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.051448 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 14:29:40.051691 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.051648 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 14:29:40.051837 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.051817 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 14:29:40.051957 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.051871 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 14:29:40.054517 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.054367 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 14:29:40.057289 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.057270 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 14:29:40.062637 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.062615 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 14:29:40.118272 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118244 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118272 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118271 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118495 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118288 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-config\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118495 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118303 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118495 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118320 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118495 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118448 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118495 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118485 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118667 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118513 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118667 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118612 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118667 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118638 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zrnd\" (UniqueName: \"kubernetes.io/projected/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-kube-api-access-4zrnd\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118667 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118658 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-web-config\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118805 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118683 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118805 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118726 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118805 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118760 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118805 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118790 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118952 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118837 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.118952 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.118943 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-config-out\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.119052 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.119014 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.119157 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.119142 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.119220 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.119181 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.219877 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.219841 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.219877 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.219878 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220141 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.219895 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220141 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.219914 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220141 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.219928 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zrnd\" (UniqueName: \"kubernetes.io/projected/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-kube-api-access-4zrnd\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220141 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220058 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-web-config\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220141 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220140 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220416 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220165 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220416 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220188 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220416 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220215 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220416 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220260 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220416 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220336 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-config-out\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220416 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220375 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220416 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220406 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220907 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220437 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220907 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220484 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220907 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220510 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220907 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220542 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-config\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220907 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220567 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220907 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220592 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.220907 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.220852 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.221331 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.221309 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.221541 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.221518 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.221655 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.221525 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.223233 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.223206 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-web-config\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.223233 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.223219 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.223747 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.223646 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.223747 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.223734 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.223886 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.223818 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.223997 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.223973 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-config-out\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.224309 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.224283 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.224416 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.224378 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.224502 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.224471 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.224730 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.224712 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.224824 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.224801 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.226555 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.226284 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.226555 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.226400 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-config\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.226708 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.226573 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.227780 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.227748 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.231017 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.229254 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zrnd\" (UniqueName: \"kubernetes.io/projected/1361e560-3fa4-43f8-96fb-23eb94bb9ce4-kube-api-access-4zrnd\") pod \"prometheus-k8s-0\" (UID: \"1361e560-3fa4-43f8-96fb-23eb94bb9ce4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.358503 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.358423 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:29:40.498382 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.498346 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 14:29:40.500372 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:29:40.500337 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1361e560_3fa4_43f8_96fb_23eb94bb9ce4.slice/crio-c3085a5329e260efc88f18a6b52802aa5df49cbae115fd28ed10ae8f70daef29 WatchSource:0}: Error finding container c3085a5329e260efc88f18a6b52802aa5df49cbae115fd28ed10ae8f70daef29: Status 404 returned error can't find the container with id c3085a5329e260efc88f18a6b52802aa5df49cbae115fd28ed10ae8f70daef29 Jul 07 14:29:40.993538 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.993508 2555 generic.go:356] "Generic (PLEG): container finished" podID="1361e560-3fa4-43f8-96fb-23eb94bb9ce4" containerID="7032884a08868787d406b5dccfa4adb2b0376ff12e594e6a712109721212c45c" exitCode=0 Jul 07 14:29:40.993941 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.993588 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1361e560-3fa4-43f8-96fb-23eb94bb9ce4","Type":"ContainerDied","Data":"7032884a08868787d406b5dccfa4adb2b0376ff12e594e6a712109721212c45c"} Jul 07 14:29:40.993941 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:40.993658 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1361e560-3fa4-43f8-96fb-23eb94bb9ce4","Type":"ContainerStarted","Data":"c3085a5329e260efc88f18a6b52802aa5df49cbae115fd28ed10ae8f70daef29"} Jul 07 14:29:41.184006 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:41.183861 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e300535-55b4-4e26-ab19-336e55d21a7e" path="/var/lib/kubelet/pods/3e300535-55b4-4e26-ab19-336e55d21a7e/volumes" Jul 07 14:29:42.002894 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:42.002862 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1361e560-3fa4-43f8-96fb-23eb94bb9ce4","Type":"ContainerStarted","Data":"300b83b0bd932bded1f62c792ef4e465db3ff1b459af69696409daf8edf48dd8"} Jul 07 14:29:42.003282 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:42.002924 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1361e560-3fa4-43f8-96fb-23eb94bb9ce4","Type":"ContainerStarted","Data":"d42e188935c772b72459f99320653ef2f760a307c8db94f0b6c42d1dd6f5e861"} Jul 07 14:29:42.003282 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:42.002954 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1361e560-3fa4-43f8-96fb-23eb94bb9ce4","Type":"ContainerStarted","Data":"c7760d5fa901a74d02de1dc7706093a8b4f3e48fb3bab33034ab096e7be22606"} Jul 07 14:29:42.003282 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:42.002986 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1361e560-3fa4-43f8-96fb-23eb94bb9ce4","Type":"ContainerStarted","Data":"17acaac81c5d7825eb92b4bc6eb1316c568dcf3134ad2df0d186312d7761716a"} Jul 07 14:29:42.003282 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:42.003011 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1361e560-3fa4-43f8-96fb-23eb94bb9ce4","Type":"ContainerStarted","Data":"6c739c118a63941d2026ea882a1a25224bbabe24c4a64ccc0171407a3a99a0e5"} Jul 07 14:29:42.003282 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:42.003043 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1361e560-3fa4-43f8-96fb-23eb94bb9ce4","Type":"ContainerStarted","Data":"3a2fa2b56076d5c4e8bef94ce5e57f193c61a33f08ceb97556067ca36444c05e"} Jul 07 14:29:42.030781 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:42.030722 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.011669387 podStartE2EDuration="2.030705361s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.035974ms" isStatefulPod=false podCreationTimestamp="2026-07-07 14:29:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:29:42.029931557 +0000 UTC m=+183.377985110" watchObservedRunningTime="2026-07-07 14:29:42.030705361 +0000 UTC m=+183.378758910" Jul 07 14:29:45.358701 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:29:45.358665 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:30:40.358939 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:30:40.358898 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:30:40.374805 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:30:40.374782 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:30:41.240568 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:30:41.240543 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 14:31:39.023377 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:31:39.023334 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:31:39.026753 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:31:39.026731 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:31:39.035806 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:31:39.035783 2555 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 14:32:43.594494 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.594416 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w"] Jul 07 14:32:43.597518 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.597502 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.600060 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.600038 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 14:32:43.600188 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.600060 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 14:32:43.600475 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.600461 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 14:32:43.600521 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.600495 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-g2dhx\"" Jul 07 14:32:43.604444 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.604413 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 14:32:43.635723 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.635696 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w"] Jul 07 14:32:43.667386 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.667358 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8kz2\" (UniqueName: \"kubernetes.io/projected/67b654c2-802f-47eb-b6cd-dcb657b48043-kube-api-access-p8kz2\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-mcp4w\" (UID: \"67b654c2-802f-47eb-b6cd-dcb657b48043\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.667533 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.667404 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67b654c2-802f-47eb-b6cd-dcb657b48043-tmp\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-mcp4w\" (UID: \"67b654c2-802f-47eb-b6cd-dcb657b48043\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.667533 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.667442 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67b654c2-802f-47eb-b6cd-dcb657b48043-webhook-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-mcp4w\" (UID: \"67b654c2-802f-47eb-b6cd-dcb657b48043\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.667533 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.667473 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/67b654c2-802f-47eb-b6cd-dcb657b48043-manifests\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-mcp4w\" (UID: \"67b654c2-802f-47eb-b6cd-dcb657b48043\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.667533 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.667487 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/67b654c2-802f-47eb-b6cd-dcb657b48043-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-mcp4w\" (UID: \"67b654c2-802f-47eb-b6cd-dcb657b48043\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.768918 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.768887 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/67b654c2-802f-47eb-b6cd-dcb657b48043-manifests\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-mcp4w\" (UID: \"67b654c2-802f-47eb-b6cd-dcb657b48043\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.768918 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.768921 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/67b654c2-802f-47eb-b6cd-dcb657b48043-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-mcp4w\" (UID: \"67b654c2-802f-47eb-b6cd-dcb657b48043\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.769206 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.769062 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8kz2\" (UniqueName: \"kubernetes.io/projected/67b654c2-802f-47eb-b6cd-dcb657b48043-kube-api-access-p8kz2\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-mcp4w\" (UID: \"67b654c2-802f-47eb-b6cd-dcb657b48043\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.769206 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.769169 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67b654c2-802f-47eb-b6cd-dcb657b48043-tmp\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-mcp4w\" (UID: \"67b654c2-802f-47eb-b6cd-dcb657b48043\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.769314 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.769252 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67b654c2-802f-47eb-b6cd-dcb657b48043-webhook-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-mcp4w\" (UID: \"67b654c2-802f-47eb-b6cd-dcb657b48043\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.769314 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.769282 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/67b654c2-802f-47eb-b6cd-dcb657b48043-manifests\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-mcp4w\" (UID: \"67b654c2-802f-47eb-b6cd-dcb657b48043\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.771435 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.771411 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67b654c2-802f-47eb-b6cd-dcb657b48043-tmp\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-mcp4w\" (UID: \"67b654c2-802f-47eb-b6cd-dcb657b48043\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.771643 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.771624 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/67b654c2-802f-47eb-b6cd-dcb657b48043-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-mcp4w\" (UID: \"67b654c2-802f-47eb-b6cd-dcb657b48043\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.771684 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.771624 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67b654c2-802f-47eb-b6cd-dcb657b48043-webhook-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-mcp4w\" (UID: \"67b654c2-802f-47eb-b6cd-dcb657b48043\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.780435 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.780407 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8kz2\" (UniqueName: \"kubernetes.io/projected/67b654c2-802f-47eb-b6cd-dcb657b48043-kube-api-access-p8kz2\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-mcp4w\" (UID: \"67b654c2-802f-47eb-b6cd-dcb657b48043\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:43.907724 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:43.907630 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:44.065497 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:44.065413 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w"] Jul 07 14:32:44.069992 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:32:44.069962 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b654c2_802f_47eb_b6cd_dcb657b48043.slice/crio-a34566cb80d0031b4667bb1cf95c1f631d75b17b6d0d05ebb7ac398387220158 WatchSource:0}: Error finding container a34566cb80d0031b4667bb1cf95c1f631d75b17b6d0d05ebb7ac398387220158: Status 404 returned error can't find the container with id a34566cb80d0031b4667bb1cf95c1f631d75b17b6d0d05ebb7ac398387220158 Jul 07 14:32:44.071725 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:44.071708 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 14:32:44.698705 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:44.698674 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" event={"ID":"67b654c2-802f-47eb-b6cd-dcb657b48043","Type":"ContainerStarted","Data":"a34566cb80d0031b4667bb1cf95c1f631d75b17b6d0d05ebb7ac398387220158"} Jul 07 14:32:48.717590 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:48.717551 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" event={"ID":"67b654c2-802f-47eb-b6cd-dcb657b48043","Type":"ContainerStarted","Data":"01a06abdb974d014d436c5d62ee6849ccd8017c6d1b04b94ce7420953a08b3cd"} Jul 07 14:32:49.723126 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:49.723091 2555 generic.go:356] "Generic (PLEG): container finished" podID="67b654c2-802f-47eb-b6cd-dcb657b48043" containerID="01a06abdb974d014d436c5d62ee6849ccd8017c6d1b04b94ce7420953a08b3cd" exitCode=0 Jul 07 14:32:49.723591 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:49.723164 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" event={"ID":"67b654c2-802f-47eb-b6cd-dcb657b48043","Type":"ContainerDied","Data":"01a06abdb974d014d436c5d62ee6849ccd8017c6d1b04b94ce7420953a08b3cd"} Jul 07 14:32:51.733743 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:51.733702 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" event={"ID":"67b654c2-802f-47eb-b6cd-dcb657b48043","Type":"ContainerStarted","Data":"6af2b9ddb7804fc2276659e4419b8eee36cc183bbf5ac7fc8d8c11aa9e365377"} Jul 07 14:32:51.734226 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:51.733849 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:32:51.757134 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:32:51.757089 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" podStartSLOduration=3.028228932 podStartE2EDuration="8.757057008s" totalImagesPullingTime="4.694826105s" totalInitContainerRuntime="1.034001971s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:32:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:32:51.755410262 +0000 UTC m=+373.103463815" watchObservedRunningTime="2026-07-07 14:32:51.757057008 +0000 UTC m=+373.105110560" Jul 07 14:33:02.740136 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:02.740100 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-mcp4w" Jul 07 14:33:12.590832 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.590796 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j"] Jul 07 14:33:12.599651 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.599621 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.601957 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.601932 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fxhgx\"" Jul 07 14:33:12.602198 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.602182 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 14:33:12.602566 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.602540 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 14:33:12.602678 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.602599 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 14:33:12.609027 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.609006 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j"] Jul 07 14:33:12.714629 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.714595 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.714629 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.714632 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.714887 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.714660 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.714887 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.714679 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.714887 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.714742 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.714887 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.714821 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.714887 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.714852 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.715104 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.714894 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.715104 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.714996 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b5hf\" (UniqueName: \"kubernetes.io/projected/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-kube-api-access-6b5hf\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.815760 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.815726 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.815959 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.815769 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.815959 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.815798 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.815959 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.815844 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.816152 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.816048 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.816152 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.816131 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.816255 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.816182 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.816255 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.816206 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b5hf\" (UniqueName: \"kubernetes.io/projected/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-kube-api-access-6b5hf\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.816397 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.816376 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.816452 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.816426 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.816506 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.816456 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.816506 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.816485 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.816769 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.816554 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.816769 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.816743 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.818326 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.818297 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.818980 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.818960 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.828524 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.828499 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b5hf\" (UniqueName: \"kubernetes.io/projected/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-kube-api-access-6b5hf\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.830336 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.830315 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787575j\" (UID: \"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:12.911549 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:12.911483 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:13.074585 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:33:13.074557 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9720233d_2f4b_4f8e_b5f9_04ba9a8bcf21.slice/crio-e60827667e6d22bbd556e3dfde5a4d8b88c903d4a31f30167d259af851ecb3b0 WatchSource:0}: Error finding container e60827667e6d22bbd556e3dfde5a4d8b88c903d4a31f30167d259af851ecb3b0: Status 404 returned error can't find the container with id e60827667e6d22bbd556e3dfde5a4d8b88c903d4a31f30167d259af851ecb3b0 Jul 07 14:33:13.077410 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:13.077389 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j"] Jul 07 14:33:13.824787 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:13.824751 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" event={"ID":"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21","Type":"ContainerStarted","Data":"e60827667e6d22bbd556e3dfde5a4d8b88c903d4a31f30167d259af851ecb3b0"} Jul 07 14:33:14.057497 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.057466 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl"] Jul 07 14:33:14.061739 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.061722 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:14.065435 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.065385 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lxhqm\"" Jul 07 14:33:14.065863 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.065844 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 14:33:14.066058 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.066040 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 14:33:14.066432 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.066413 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 14:33:14.066610 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.066590 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 14:33:14.067060 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.067043 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 14:33:14.077538 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.077489 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl"] Jul 07 14:33:14.130929 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.130903 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75p2c\" (UniqueName: \"kubernetes.io/projected/eaa03334-56df-42c4-9d42-05ac60413b2b-kube-api-access-75p2c\") pod \"lws-controller-manager-849574fc57-vrvsl\" (UID: \"eaa03334-56df-42c4-9d42-05ac60413b2b\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:14.131105 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.130957 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eaa03334-56df-42c4-9d42-05ac60413b2b-manager-config\") pod \"lws-controller-manager-849574fc57-vrvsl\" (UID: \"eaa03334-56df-42c4-9d42-05ac60413b2b\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:14.131105 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.130985 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eaa03334-56df-42c4-9d42-05ac60413b2b-metrics-cert\") pod \"lws-controller-manager-849574fc57-vrvsl\" (UID: \"eaa03334-56df-42c4-9d42-05ac60413b2b\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:14.131105 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.131024 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eaa03334-56df-42c4-9d42-05ac60413b2b-cert\") pod \"lws-controller-manager-849574fc57-vrvsl\" (UID: \"eaa03334-56df-42c4-9d42-05ac60413b2b\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:14.232098 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.232031 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eaa03334-56df-42c4-9d42-05ac60413b2b-manager-config\") pod \"lws-controller-manager-849574fc57-vrvsl\" (UID: \"eaa03334-56df-42c4-9d42-05ac60413b2b\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:14.232355 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.232104 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eaa03334-56df-42c4-9d42-05ac60413b2b-metrics-cert\") pod \"lws-controller-manager-849574fc57-vrvsl\" (UID: \"eaa03334-56df-42c4-9d42-05ac60413b2b\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:14.232355 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.232148 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eaa03334-56df-42c4-9d42-05ac60413b2b-cert\") pod \"lws-controller-manager-849574fc57-vrvsl\" (UID: \"eaa03334-56df-42c4-9d42-05ac60413b2b\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:14.232355 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.232251 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75p2c\" (UniqueName: \"kubernetes.io/projected/eaa03334-56df-42c4-9d42-05ac60413b2b-kube-api-access-75p2c\") pod \"lws-controller-manager-849574fc57-vrvsl\" (UID: \"eaa03334-56df-42c4-9d42-05ac60413b2b\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:14.233377 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.233328 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eaa03334-56df-42c4-9d42-05ac60413b2b-manager-config\") pod \"lws-controller-manager-849574fc57-vrvsl\" (UID: \"eaa03334-56df-42c4-9d42-05ac60413b2b\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:14.236025 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.235998 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eaa03334-56df-42c4-9d42-05ac60413b2b-metrics-cert\") pod \"lws-controller-manager-849574fc57-vrvsl\" (UID: \"eaa03334-56df-42c4-9d42-05ac60413b2b\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:14.236483 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.236461 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eaa03334-56df-42c4-9d42-05ac60413b2b-cert\") pod \"lws-controller-manager-849574fc57-vrvsl\" (UID: \"eaa03334-56df-42c4-9d42-05ac60413b2b\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:14.241405 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.241381 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75p2c\" (UniqueName: \"kubernetes.io/projected/eaa03334-56df-42c4-9d42-05ac60413b2b-kube-api-access-75p2c\") pod \"lws-controller-manager-849574fc57-vrvsl\" (UID: \"eaa03334-56df-42c4-9d42-05ac60413b2b\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:14.372503 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.372415 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:14.656734 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.656652 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl"] Jul 07 14:33:14.660772 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:33:14.660743 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaa03334_56df_42c4_9d42_05ac60413b2b.slice/crio-c0acde04dbe21d5234cf4a95baed04b20259a530336c9ad836f57d23da1ee0cf WatchSource:0}: Error finding container c0acde04dbe21d5234cf4a95baed04b20259a530336c9ad836f57d23da1ee0cf: Status 404 returned error can't find the container with id c0acde04dbe21d5234cf4a95baed04b20259a530336c9ad836f57d23da1ee0cf Jul 07 14:33:14.831332 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:14.831295 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" event={"ID":"eaa03334-56df-42c4-9d42-05ac60413b2b","Type":"ContainerStarted","Data":"c0acde04dbe21d5234cf4a95baed04b20259a530336c9ad836f57d23da1ee0cf"} Jul 07 14:33:15.958751 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:15.957146 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4s65s"] Jul 07 14:33:15.984003 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:15.983967 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4"] Jul 07 14:33:15.984173 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:15.984111 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4s65s" Jul 07 14:33:15.988082 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:15.988038 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-xklj5\"" Jul 07 14:33:15.988461 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:15.988441 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 07 14:33:15.989269 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:15.989247 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4" Jul 07 14:33:15.992741 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:15.992720 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 07 14:33:15.993479 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:15.993257 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4s65s"] Jul 07 14:33:16.000586 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.000552 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wglrn\"" Jul 07 14:33:16.043773 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.043744 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4"] Jul 07 14:33:16.050521 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.050490 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z57f\" (UniqueName: \"kubernetes.io/projected/9ef112f3-0a1b-4e2f-b925-79e9455ea0c4-kube-api-access-7z57f\") pod \"kserve-controller-manager-d6f4b65b8-4s65s\" (UID: \"9ef112f3-0a1b-4e2f-b925-79e9455ea0c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4s65s" Jul 07 14:33:16.050670 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.050546 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2929b9cf-77eb-43d9-aa9a-7f115af06b0c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xbzf4\" (UID: \"2929b9cf-77eb-43d9-aa9a-7f115af06b0c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4" Jul 07 14:33:16.050670 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.050620 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p68sz\" (UniqueName: \"kubernetes.io/projected/2929b9cf-77eb-43d9-aa9a-7f115af06b0c-kube-api-access-p68sz\") pod \"llmisvc-controller-manager-777c6bd7d5-xbzf4\" (UID: \"2929b9cf-77eb-43d9-aa9a-7f115af06b0c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4" Jul 07 14:33:16.050774 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.050712 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ef112f3-0a1b-4e2f-b925-79e9455ea0c4-cert\") pod \"kserve-controller-manager-d6f4b65b8-4s65s\" (UID: \"9ef112f3-0a1b-4e2f-b925-79e9455ea0c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4s65s" Jul 07 14:33:16.124120 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.124061 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:33:16.124241 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.124158 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:33:16.124241 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.124187 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:33:16.151408 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.151362 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ef112f3-0a1b-4e2f-b925-79e9455ea0c4-cert\") pod \"kserve-controller-manager-d6f4b65b8-4s65s\" (UID: \"9ef112f3-0a1b-4e2f-b925-79e9455ea0c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4s65s" Jul 07 14:33:16.151545 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.151487 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z57f\" (UniqueName: \"kubernetes.io/projected/9ef112f3-0a1b-4e2f-b925-79e9455ea0c4-kube-api-access-7z57f\") pod \"kserve-controller-manager-d6f4b65b8-4s65s\" (UID: \"9ef112f3-0a1b-4e2f-b925-79e9455ea0c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4s65s" Jul 07 14:33:16.151545 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.151527 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2929b9cf-77eb-43d9-aa9a-7f115af06b0c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xbzf4\" (UID: \"2929b9cf-77eb-43d9-aa9a-7f115af06b0c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4" Jul 07 14:33:16.151700 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.151568 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p68sz\" (UniqueName: \"kubernetes.io/projected/2929b9cf-77eb-43d9-aa9a-7f115af06b0c-kube-api-access-p68sz\") pod \"llmisvc-controller-manager-777c6bd7d5-xbzf4\" (UID: \"2929b9cf-77eb-43d9-aa9a-7f115af06b0c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4" Jul 07 14:33:16.154914 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.154864 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ef112f3-0a1b-4e2f-b925-79e9455ea0c4-cert\") pod \"kserve-controller-manager-d6f4b65b8-4s65s\" (UID: \"9ef112f3-0a1b-4e2f-b925-79e9455ea0c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4s65s" Jul 07 14:33:16.156257 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.156238 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2929b9cf-77eb-43d9-aa9a-7f115af06b0c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xbzf4\" (UID: \"2929b9cf-77eb-43d9-aa9a-7f115af06b0c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4" Jul 07 14:33:16.168725 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.168663 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p68sz\" (UniqueName: \"kubernetes.io/projected/2929b9cf-77eb-43d9-aa9a-7f115af06b0c-kube-api-access-p68sz\") pod \"llmisvc-controller-manager-777c6bd7d5-xbzf4\" (UID: \"2929b9cf-77eb-43d9-aa9a-7f115af06b0c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4" Jul 07 14:33:16.169010 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.168987 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z57f\" (UniqueName: \"kubernetes.io/projected/9ef112f3-0a1b-4e2f-b925-79e9455ea0c4-kube-api-access-7z57f\") pod \"kserve-controller-manager-d6f4b65b8-4s65s\" (UID: \"9ef112f3-0a1b-4e2f-b925-79e9455ea0c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4s65s" Jul 07 14:33:16.298659 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.298487 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4s65s" Jul 07 14:33:16.307098 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.307040 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4" Jul 07 14:33:16.541157 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.541133 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4s65s"] Jul 07 14:33:16.544878 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:33:16.544843 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef112f3_0a1b_4e2f_b925_79e9455ea0c4.slice/crio-0a02d29d8eb0d81b6cc58cee8beb6f03d5cf738baabed950302ce72041c9cc10 WatchSource:0}: Error finding container 0a02d29d8eb0d81b6cc58cee8beb6f03d5cf738baabed950302ce72041c9cc10: Status 404 returned error can't find the container with id 0a02d29d8eb0d81b6cc58cee8beb6f03d5cf738baabed950302ce72041c9cc10 Jul 07 14:33:16.554017 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.553399 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4"] Jul 07 14:33:16.556486 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:33:16.556460 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2929b9cf_77eb_43d9_aa9a_7f115af06b0c.slice/crio-f490057f7b28d6cbce1d67fbd131f820da6fa07781b288dca069d61ee3ad5352 WatchSource:0}: Error finding container f490057f7b28d6cbce1d67fbd131f820da6fa07781b288dca069d61ee3ad5352: Status 404 returned error can't find the container with id f490057f7b28d6cbce1d67fbd131f820da6fa07781b288dca069d61ee3ad5352 Jul 07 14:33:16.847226 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.847177 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4" event={"ID":"2929b9cf-77eb-43d9-aa9a-7f115af06b0c","Type":"ContainerStarted","Data":"f490057f7b28d6cbce1d67fbd131f820da6fa07781b288dca069d61ee3ad5352"} Jul 07 14:33:16.849052 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.849018 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" event={"ID":"9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21","Type":"ContainerStarted","Data":"1266db93e20bf4768f10b30cafcab341619666f934b49dcaf402d88ee03f8972"} Jul 07 14:33:16.850402 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.850366 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4s65s" event={"ID":"9ef112f3-0a1b-4e2f-b925-79e9455ea0c4","Type":"ContainerStarted","Data":"0a02d29d8eb0d81b6cc58cee8beb6f03d5cf738baabed950302ce72041c9cc10"} Jul 07 14:33:16.878586 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.877939 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" podStartSLOduration=1.831094598 podStartE2EDuration="4.877919239s" totalImagesPullingTime="3.046824641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:16.877341784 +0000 UTC m=+398.225395338" watchObservedRunningTime="2026-07-07 14:33:16.877919239 +0000 UTC m=+398.225972776" Jul 07 14:33:16.912552 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:16.912523 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:17.917762 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:17.917732 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:18.865875 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:18.865839 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" event={"ID":"eaa03334-56df-42c4-9d42-05ac60413b2b","Type":"ContainerStarted","Data":"96dd8a4bb6e379f726539f677c04cb80559c948c61c511533b284981c40e5b67"} Jul 07 14:33:18.866046 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:18.866038 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:18.866136 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:18.866083 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:18.867313 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:18.867268 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787575j" Jul 07 14:33:18.884670 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:18.884628 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" podStartSLOduration=1.515267788 podStartE2EDuration="4.884613215s" totalImagesPullingTime="3.369345427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:18.882266311 +0000 UTC m=+400.230319864" watchObservedRunningTime="2026-07-07 14:33:18.884613215 +0000 UTC m=+400.232666768" Jul 07 14:33:20.876936 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:20.876897 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4s65s" event={"ID":"9ef112f3-0a1b-4e2f-b925-79e9455ea0c4","Type":"ContainerStarted","Data":"1243af9ea56308d1f939ef3cd431404586e977eb9d72e5ec56ddb4d94631c8f1"} Jul 07 14:33:20.877435 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:20.877104 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4s65s" Jul 07 14:33:20.894690 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:20.894646 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4s65s" podStartSLOduration=2.192126981 podStartE2EDuration="5.894629346s" totalImagesPullingTime="3.702502365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:20.893657298 +0000 UTC m=+402.241710852" watchObservedRunningTime="2026-07-07 14:33:20.894629346 +0000 UTC m=+402.242682900" Jul 07 14:33:21.883466 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:21.883434 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4" event={"ID":"2929b9cf-77eb-43d9-aa9a-7f115af06b0c","Type":"ContainerStarted","Data":"62462f14806e1dcdf4e0fee4cac739250d303828c94ea6dbc5e80297e34dfe83"} Jul 07 14:33:21.883834 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:21.883553 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4" Jul 07 14:33:21.900515 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:21.900468 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4" podStartSLOduration=2.309777506 podStartE2EDuration="6.900451825s" totalImagesPullingTime="4.590674319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:21.899269954 +0000 UTC m=+403.247323506" watchObservedRunningTime="2026-07-07 14:33:21.900451825 +0000 UTC m=+403.248505410" Jul 07 14:33:29.872204 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:29.872174 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-849574fc57-vrvsl" Jul 07 14:33:51.888699 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:51.888669 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4s65s" Jul 07 14:33:52.890236 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:33:52.890209 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xbzf4" Jul 07 14:34:42.853443 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:42.853409 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8"] Jul 07 14:34:42.855661 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:42.855646 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:34:42.858089 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:42.858049 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 14:34:42.858540 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:42.858522 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f2kgf\"" Jul 07 14:34:42.858623 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:42.858566 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 14:34:42.909603 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:42.909576 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8"] Jul 07 14:34:42.914487 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:42.914458 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7652c479-a55a-40f0-9dd4-7e7c12ca0162-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-thjw8\" (UID: \"7652c479-a55a-40f0-9dd4-7e7c12ca0162\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:34:42.914642 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:42.914509 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8q2q\" (UniqueName: \"kubernetes.io/projected/7652c479-a55a-40f0-9dd4-7e7c12ca0162-kube-api-access-r8q2q\") pod \"kuadrant-operator-controller-manager-8684c9ff49-thjw8\" (UID: \"7652c479-a55a-40f0-9dd4-7e7c12ca0162\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:34:43.015756 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:43.015719 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7652c479-a55a-40f0-9dd4-7e7c12ca0162-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-thjw8\" (UID: \"7652c479-a55a-40f0-9dd4-7e7c12ca0162\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:34:43.015900 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:43.015760 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8q2q\" (UniqueName: \"kubernetes.io/projected/7652c479-a55a-40f0-9dd4-7e7c12ca0162-kube-api-access-r8q2q\") pod \"kuadrant-operator-controller-manager-8684c9ff49-thjw8\" (UID: \"7652c479-a55a-40f0-9dd4-7e7c12ca0162\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:34:43.016123 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:43.016100 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7652c479-a55a-40f0-9dd4-7e7c12ca0162-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-thjw8\" (UID: \"7652c479-a55a-40f0-9dd4-7e7c12ca0162\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:34:43.032216 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:43.032193 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8q2q\" (UniqueName: \"kubernetes.io/projected/7652c479-a55a-40f0-9dd4-7e7c12ca0162-kube-api-access-r8q2q\") pod \"kuadrant-operator-controller-manager-8684c9ff49-thjw8\" (UID: \"7652c479-a55a-40f0-9dd4-7e7c12ca0162\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:34:43.165658 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:43.165567 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:34:43.302118 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:43.302084 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8"] Jul 07 14:34:43.306647 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:34:43.306620 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7652c479_a55a_40f0_9dd4_7e7c12ca0162.slice/crio-a3618ac3e0b88288adea7dba5b608a6694ca239362ea1c7eefb38b8d52f966b9 WatchSource:0}: Error finding container a3618ac3e0b88288adea7dba5b608a6694ca239362ea1c7eefb38b8d52f966b9: Status 404 returned error can't find the container with id a3618ac3e0b88288adea7dba5b608a6694ca239362ea1c7eefb38b8d52f966b9 Jul 07 14:34:44.228532 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:44.228488 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" event={"ID":"7652c479-a55a-40f0-9dd4-7e7c12ca0162","Type":"ContainerStarted","Data":"a3618ac3e0b88288adea7dba5b608a6694ca239362ea1c7eefb38b8d52f966b9"} Jul 07 14:34:48.832851 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:48.832814 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-psxch"] Jul 07 14:34:48.836405 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:48.836383 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psxch" Jul 07 14:34:48.838425 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:48.838405 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 07 14:34:48.838538 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:48.838445 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-zwfgw\"" Jul 07 14:34:48.847487 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:48.847463 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-psxch"] Jul 07 14:34:48.870952 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:48.870921 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xw44\" (UniqueName: \"kubernetes.io/projected/91d482f7-422e-459d-a537-0349e791fa0e-kube-api-access-8xw44\") pod \"dns-operator-controller-manager-864dd78c4c-psxch\" (UID: \"91d482f7-422e-459d-a537-0349e791fa0e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psxch" Jul 07 14:34:48.971736 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:48.971692 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xw44\" (UniqueName: \"kubernetes.io/projected/91d482f7-422e-459d-a537-0349e791fa0e-kube-api-access-8xw44\") pod \"dns-operator-controller-manager-864dd78c4c-psxch\" (UID: \"91d482f7-422e-459d-a537-0349e791fa0e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psxch" Jul 07 14:34:48.979908 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:48.979875 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xw44\" (UniqueName: \"kubernetes.io/projected/91d482f7-422e-459d-a537-0349e791fa0e-kube-api-access-8xw44\") pod \"dns-operator-controller-manager-864dd78c4c-psxch\" (UID: \"91d482f7-422e-459d-a537-0349e791fa0e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psxch" Jul 07 14:34:49.146721 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:49.146636 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psxch" Jul 07 14:34:49.288051 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:49.288024 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-psxch"] Jul 07 14:34:49.290779 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:34:49.290750 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91d482f7_422e_459d_a537_0349e791fa0e.slice/crio-96c4253611952edb6ff1f8301e2a26593434d5a2299a9666ff8764b9649bfdae WatchSource:0}: Error finding container 96c4253611952edb6ff1f8301e2a26593434d5a2299a9666ff8764b9649bfdae: Status 404 returned error can't find the container with id 96c4253611952edb6ff1f8301e2a26593434d5a2299a9666ff8764b9649bfdae Jul 07 14:34:50.260149 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:50.260108 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psxch" event={"ID":"91d482f7-422e-459d-a537-0349e791fa0e","Type":"ContainerStarted","Data":"96c4253611952edb6ff1f8301e2a26593434d5a2299a9666ff8764b9649bfdae"} Jul 07 14:34:51.267973 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:51.267885 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" event={"ID":"7652c479-a55a-40f0-9dd4-7e7c12ca0162","Type":"ContainerStarted","Data":"43df454e81c96d0b3e64de5bf420797ece400a63b188e09c17ebf4e7a64f3541"} Jul 07 14:34:51.268432 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:51.267993 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:34:51.292945 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:51.292881 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" podStartSLOduration=1.5897149050000001 podStartE2EDuration="9.292858021s" totalImagesPullingTime="7.703143116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:34:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:51.290165445 +0000 UTC m=+492.638219000" watchObservedRunningTime="2026-07-07 14:34:51.292858021 +0000 UTC m=+492.640911576" Jul 07 14:34:53.279266 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:53.279230 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psxch" event={"ID":"91d482f7-422e-459d-a537-0349e791fa0e","Type":"ContainerStarted","Data":"4244384c57f8348a7c3ed8bc98678f4d69342d1c96fd4913443974329621e5cb"} Jul 07 14:34:53.279655 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:53.279361 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psxch" Jul 07 14:34:53.299246 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:34:53.299205 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psxch" podStartSLOduration=1.975732971 podStartE2EDuration="5.299190406s" totalImagesPullingTime="3.323457435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:53.297530633 +0000 UTC m=+494.645584187" watchObservedRunningTime="2026-07-07 14:34:53.299190406 +0000 UTC m=+494.647243958" Jul 07 14:35:02.275588 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:02.275551 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:35:04.130322 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.130284 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx"] Jul 07 14:35:04.133865 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.133843 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" Jul 07 14:35:04.161520 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.161493 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx"] Jul 07 14:35:04.213749 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.213716 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6707a73f-cb41-4498-ba8c-549f1a8c6526-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sz9qx\" (UID: \"6707a73f-cb41-4498-ba8c-549f1a8c6526\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" Jul 07 14:35:04.213749 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.213752 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4srd\" (UniqueName: \"kubernetes.io/projected/6707a73f-cb41-4498-ba8c-549f1a8c6526-kube-api-access-z4srd\") pod \"kuadrant-operator-controller-manager-86878c769f-sz9qx\" (UID: \"6707a73f-cb41-4498-ba8c-549f1a8c6526\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" Jul 07 14:35:04.240307 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.240272 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8"] Jul 07 14:35:04.240512 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.240475 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" podUID="7652c479-a55a-40f0-9dd4-7e7c12ca0162" containerName="manager" containerID="cri-o://43df454e81c96d0b3e64de5bf420797ece400a63b188e09c17ebf4e7a64f3541" gracePeriod=2 Jul 07 14:35:04.256931 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.256904 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8"] Jul 07 14:35:04.285667 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.285637 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psxch" Jul 07 14:35:04.288638 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.288614 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx"] Jul 07 14:35:04.288884 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:04.288867 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-z4srd], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" podUID="6707a73f-cb41-4498-ba8c-549f1a8c6526" Jul 07 14:35:04.293955 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.293918 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx"] Jul 07 14:35:04.314352 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:04.314306 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-thjw8\" is forbidden: User \"system:node:ip-10-0-141-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-129.ec2.internal' and this object" podUID="7652c479-a55a-40f0-9dd4-7e7c12ca0162" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:35:04.314610 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.314587 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6707a73f-cb41-4498-ba8c-549f1a8c6526-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sz9qx\" (UID: \"6707a73f-cb41-4498-ba8c-549f1a8c6526\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" Jul 07 14:35:04.314694 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.314626 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4srd\" (UniqueName: \"kubernetes.io/projected/6707a73f-cb41-4498-ba8c-549f1a8c6526-kube-api-access-z4srd\") pod \"kuadrant-operator-controller-manager-86878c769f-sz9qx\" (UID: \"6707a73f-cb41-4498-ba8c-549f1a8c6526\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" Jul 07 14:35:04.314955 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.314938 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6707a73f-cb41-4498-ba8c-549f1a8c6526-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sz9qx\" (UID: \"6707a73f-cb41-4498-ba8c-549f1a8c6526\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" Jul 07 14:35:04.316946 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:04.316926 2555 projected.go:196] Error preparing data for projected volume kube-api-access-z4srd for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx: failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-141-129.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-141-129.ec2.internal' and this object Jul 07 14:35:04.317054 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:04.316985 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6707a73f-cb41-4498-ba8c-549f1a8c6526-kube-api-access-z4srd podName:6707a73f-cb41-4498-ba8c-549f1a8c6526 nodeName:}" failed. No retries permitted until 2026-07-07 14:35:04.816969372 +0000 UTC m=+506.165022902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-z4srd" (UniqueName: "kubernetes.io/projected/6707a73f-cb41-4498-ba8c-549f1a8c6526-kube-api-access-z4srd") pod "kuadrant-operator-controller-manager-86878c769f-sz9qx" (UID: "6707a73f-cb41-4498-ba8c-549f1a8c6526") : failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-141-129.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-141-129.ec2.internal' and this object Jul 07 14:35:04.326621 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.326593 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" Jul 07 14:35:04.334014 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.333992 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" Jul 07 14:35:04.334014 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:04.333997 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-sz9qx\" is forbidden: User \"system:node:ip-10-0-141-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-129.ec2.internal' and this object" podUID="6707a73f-cb41-4498-ba8c-549f1a8c6526" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" Jul 07 14:35:04.338686 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:04.338647 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-thjw8\" is forbidden: User \"system:node:ip-10-0-141-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-129.ec2.internal' and this object" podUID="7652c479-a55a-40f0-9dd4-7e7c12ca0162" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:35:04.342005 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:04.341974 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-sz9qx\" is forbidden: User \"system:node:ip-10-0-141-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-129.ec2.internal' and this object" podUID="6707a73f-cb41-4498-ba8c-549f1a8c6526" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" Jul 07 14:35:04.345086 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.345042 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29"] Jul 07 14:35:04.345450 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.345425 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7652c479-a55a-40f0-9dd4-7e7c12ca0162" containerName="manager" Jul 07 14:35:04.345450 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.345441 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="7652c479-a55a-40f0-9dd4-7e7c12ca0162" containerName="manager" Jul 07 14:35:04.345612 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.345507 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="7652c479-a55a-40f0-9dd4-7e7c12ca0162" containerName="manager" Jul 07 14:35:04.348502 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.348481 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" Jul 07 14:35:04.355787 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:04.355759 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-thjw8\" is forbidden: User \"system:node:ip-10-0-141-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-129.ec2.internal' and this object" podUID="7652c479-a55a-40f0-9dd4-7e7c12ca0162" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:35:04.357564 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:04.357536 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-sz9qx\" is forbidden: User \"system:node:ip-10-0-141-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-129.ec2.internal' and this object" podUID="6707a73f-cb41-4498-ba8c-549f1a8c6526" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" Jul 07 14:35:04.367913 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.367888 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mj22f"] Jul 07 14:35:04.370998 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.370981 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mj22f" Jul 07 14:35:04.380750 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:04.380698 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"limitador-operator-controller-manager-dockercfg-cpkbs\" is forbidden: User \"system:node:ip-10-0-141-129.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-129.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-cpkbs\"" type="*v1.Secret" Jul 07 14:35:04.393311 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.393277 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29"] Jul 07 14:35:04.395744 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.395719 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mj22f"] Jul 07 14:35:04.412769 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:04.412732 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-thjw8\" is forbidden: User \"system:node:ip-10-0-141-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-129.ec2.internal' and this object" podUID="7652c479-a55a-40f0-9dd4-7e7c12ca0162" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:35:04.414690 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:04.414659 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-thjw8\" is forbidden: User \"system:node:ip-10-0-141-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-129.ec2.internal' and this object" podUID="7652c479-a55a-40f0-9dd4-7e7c12ca0162" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:35:04.415091 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.415053 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6707a73f-cb41-4498-ba8c-549f1a8c6526-extensions-socket-volume\") pod \"6707a73f-cb41-4498-ba8c-549f1a8c6526\" (UID: \"6707a73f-cb41-4498-ba8c-549f1a8c6526\") " Jul 07 14:35:04.415280 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.415261 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a1e446f-7ce9-4f73-9c9a-9fb573e428f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vrp29\" (UID: \"6a1e446f-7ce9-4f73-9c9a-9fb573e428f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" Jul 07 14:35:04.415371 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.415344 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6707a73f-cb41-4498-ba8c-549f1a8c6526-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6707a73f-cb41-4498-ba8c-549f1a8c6526" (UID: "6707a73f-cb41-4498-ba8c-549f1a8c6526"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:35:04.415502 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.415479 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkp2t\" (UniqueName: \"kubernetes.io/projected/6a1e446f-7ce9-4f73-9c9a-9fb573e428f9-kube-api-access-wkp2t\") pod \"kuadrant-operator-controller-manager-86878c769f-vrp29\" (UID: \"6a1e446f-7ce9-4f73-9c9a-9fb573e428f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" Jul 07 14:35:04.415617 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.415602 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4srd\" (UniqueName: \"kubernetes.io/projected/6707a73f-cb41-4498-ba8c-549f1a8c6526-kube-api-access-z4srd\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:35:04.415664 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.415624 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6707a73f-cb41-4498-ba8c-549f1a8c6526-extensions-socket-volume\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:35:04.438237 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:04.438198 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-sz9qx\" is forbidden: User \"system:node:ip-10-0-141-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-129.ec2.internal' and this object" podUID="6707a73f-cb41-4498-ba8c-549f1a8c6526" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" Jul 07 14:35:04.498589 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.498565 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:35:04.500620 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:04.500588 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-sz9qx\" is forbidden: User \"system:node:ip-10-0-141-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-129.ec2.internal' and this object" podUID="6707a73f-cb41-4498-ba8c-549f1a8c6526" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" Jul 07 14:35:04.502234 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:04.502207 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-thjw8\" is forbidden: User \"system:node:ip-10-0-141-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-129.ec2.internal' and this object" podUID="7652c479-a55a-40f0-9dd4-7e7c12ca0162" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:35:04.516746 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.516712 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkp2t\" (UniqueName: \"kubernetes.io/projected/6a1e446f-7ce9-4f73-9c9a-9fb573e428f9-kube-api-access-wkp2t\") pod \"kuadrant-operator-controller-manager-86878c769f-vrp29\" (UID: \"6a1e446f-7ce9-4f73-9c9a-9fb573e428f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" Jul 07 14:35:04.516853 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.516780 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a1e446f-7ce9-4f73-9c9a-9fb573e428f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vrp29\" (UID: \"6a1e446f-7ce9-4f73-9c9a-9fb573e428f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" Jul 07 14:35:04.516853 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.516839 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9fnx\" (UniqueName: \"kubernetes.io/projected/2c0d0e79-7cf7-4605-9a72-7823389f4e6d-kube-api-access-n9fnx\") pod \"limitador-operator-controller-manager-859d7b78bd-mj22f\" (UID: \"2c0d0e79-7cf7-4605-9a72-7823389f4e6d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mj22f" Jul 07 14:35:04.517129 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.517112 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a1e446f-7ce9-4f73-9c9a-9fb573e428f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vrp29\" (UID: \"6a1e446f-7ce9-4f73-9c9a-9fb573e428f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" Jul 07 14:35:04.527692 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.527662 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkp2t\" (UniqueName: \"kubernetes.io/projected/6a1e446f-7ce9-4f73-9c9a-9fb573e428f9-kube-api-access-wkp2t\") pod \"kuadrant-operator-controller-manager-86878c769f-vrp29\" (UID: \"6a1e446f-7ce9-4f73-9c9a-9fb573e428f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" Jul 07 14:35:04.617495 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.617462 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7652c479-a55a-40f0-9dd4-7e7c12ca0162-extensions-socket-volume\") pod \"7652c479-a55a-40f0-9dd4-7e7c12ca0162\" (UID: \"7652c479-a55a-40f0-9dd4-7e7c12ca0162\") " Jul 07 14:35:04.617660 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.617533 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8q2q\" (UniqueName: \"kubernetes.io/projected/7652c479-a55a-40f0-9dd4-7e7c12ca0162-kube-api-access-r8q2q\") pod \"7652c479-a55a-40f0-9dd4-7e7c12ca0162\" (UID: \"7652c479-a55a-40f0-9dd4-7e7c12ca0162\") " Jul 07 14:35:04.617745 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.617695 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9fnx\" (UniqueName: \"kubernetes.io/projected/2c0d0e79-7cf7-4605-9a72-7823389f4e6d-kube-api-access-n9fnx\") pod \"limitador-operator-controller-manager-859d7b78bd-mj22f\" (UID: \"2c0d0e79-7cf7-4605-9a72-7823389f4e6d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mj22f" Jul 07 14:35:04.618043 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.618011 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7652c479-a55a-40f0-9dd4-7e7c12ca0162-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7652c479-a55a-40f0-9dd4-7e7c12ca0162" (UID: "7652c479-a55a-40f0-9dd4-7e7c12ca0162"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:35:04.619788 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.619762 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7652c479-a55a-40f0-9dd4-7e7c12ca0162-kube-api-access-r8q2q" (OuterVolumeSpecName: "kube-api-access-r8q2q") pod "7652c479-a55a-40f0-9dd4-7e7c12ca0162" (UID: "7652c479-a55a-40f0-9dd4-7e7c12ca0162"). InnerVolumeSpecName "kube-api-access-r8q2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:35:04.631012 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.630959 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9fnx\" (UniqueName: \"kubernetes.io/projected/2c0d0e79-7cf7-4605-9a72-7823389f4e6d-kube-api-access-n9fnx\") pod \"limitador-operator-controller-manager-859d7b78bd-mj22f\" (UID: \"2c0d0e79-7cf7-4605-9a72-7823389f4e6d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mj22f" Jul 07 14:35:04.659104 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.659056 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" Jul 07 14:35:04.718712 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.718682 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8q2q\" (UniqueName: \"kubernetes.io/projected/7652c479-a55a-40f0-9dd4-7e7c12ca0162-kube-api-access-r8q2q\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:35:04.718712 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.718710 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7652c479-a55a-40f0-9dd4-7e7c12ca0162-extensions-socket-volume\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:35:04.831781 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:04.831746 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29"] Jul 07 14:35:04.835411 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:35:04.835384 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a1e446f_7ce9_4f73_9c9a_9fb573e428f9.slice/crio-d639761478d1612435fbeb93f814ab630359c6c59247fb2a5120f9ed3e8bb010 WatchSource:0}: Error finding container d639761478d1612435fbeb93f814ab630359c6c59247fb2a5120f9ed3e8bb010: Status 404 returned error can't find the container with id d639761478d1612435fbeb93f814ab630359c6c59247fb2a5120f9ed3e8bb010 Jul 07 14:35:05.179943 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:05.179865 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6707a73f-cb41-4498-ba8c-549f1a8c6526" path="/var/lib/kubelet/pods/6707a73f-cb41-4498-ba8c-549f1a8c6526/volumes" Jul 07 14:35:05.180301 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:05.180132 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7652c479-a55a-40f0-9dd4-7e7c12ca0162" path="/var/lib/kubelet/pods/7652c479-a55a-40f0-9dd4-7e7c12ca0162/volumes" Jul 07 14:35:05.332244 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:05.332212 2555 generic.go:356] "Generic (PLEG): container finished" podID="7652c479-a55a-40f0-9dd4-7e7c12ca0162" containerID="43df454e81c96d0b3e64de5bf420797ece400a63b188e09c17ebf4e7a64f3541" exitCode=0 Jul 07 14:35:05.332413 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:05.332258 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-thjw8" Jul 07 14:35:05.332413 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:05.332311 2555 scope.go:117] "RemoveContainer" containerID="43df454e81c96d0b3e64de5bf420797ece400a63b188e09c17ebf4e7a64f3541" Jul 07 14:35:05.334190 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:05.334158 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" Jul 07 14:35:05.334318 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:05.334187 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" event={"ID":"6a1e446f-7ce9-4f73-9c9a-9fb573e428f9","Type":"ContainerStarted","Data":"2ebd3a641edb3b07a2cdcaa7e243e507f4d9e641142724e2b6cbac0c6da0f573"} Jul 07 14:35:05.334318 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:05.334246 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" event={"ID":"6a1e446f-7ce9-4f73-9c9a-9fb573e428f9","Type":"ContainerStarted","Data":"d639761478d1612435fbeb93f814ab630359c6c59247fb2a5120f9ed3e8bb010"} Jul 07 14:35:05.341022 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:05.341003 2555 scope.go:117] "RemoveContainer" containerID="43df454e81c96d0b3e64de5bf420797ece400a63b188e09c17ebf4e7a64f3541" Jul 07 14:35:05.341354 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:05.341333 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43df454e81c96d0b3e64de5bf420797ece400a63b188e09c17ebf4e7a64f3541\": container with ID starting with 43df454e81c96d0b3e64de5bf420797ece400a63b188e09c17ebf4e7a64f3541 not found: ID does not exist" containerID="43df454e81c96d0b3e64de5bf420797ece400a63b188e09c17ebf4e7a64f3541" Jul 07 14:35:05.341432 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:05.341361 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43df454e81c96d0b3e64de5bf420797ece400a63b188e09c17ebf4e7a64f3541"} err="failed to get container status \"43df454e81c96d0b3e64de5bf420797ece400a63b188e09c17ebf4e7a64f3541\": rpc error: code = NotFound desc = could not find container \"43df454e81c96d0b3e64de5bf420797ece400a63b188e09c17ebf4e7a64f3541\": container with ID starting with 43df454e81c96d0b3e64de5bf420797ece400a63b188e09c17ebf4e7a64f3541 not found: ID does not exist" Jul 07 14:35:05.379454 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:35:05.379416 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-sz9qx\" is forbidden: User \"system:node:ip-10-0-141-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-129.ec2.internal' and this object" podUID="6707a73f-cb41-4498-ba8c-549f1a8c6526" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sz9qx" Jul 07 14:35:05.512937 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:05.512904 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-cpkbs\"" Jul 07 14:35:05.522360 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:05.522330 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mj22f" Jul 07 14:35:05.677602 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:05.677512 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" podStartSLOduration=1.677491717 podStartE2EDuration="1.677491717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:35:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:35:05.377468902 +0000 UTC m=+506.725522457" watchObservedRunningTime="2026-07-07 14:35:05.677491717 +0000 UTC m=+507.025545274" Jul 07 14:35:05.677740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:05.677723 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mj22f"] Jul 07 14:35:05.680702 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:35:05.680669 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c0d0e79_7cf7_4605_9a72_7823389f4e6d.slice/crio-5499f7b103cff2b94e3801b80c47f0d3499ef3b92627d8cfbbce2f30fdf166e8 WatchSource:0}: Error finding container 5499f7b103cff2b94e3801b80c47f0d3499ef3b92627d8cfbbce2f30fdf166e8: Status 404 returned error can't find the container with id 5499f7b103cff2b94e3801b80c47f0d3499ef3b92627d8cfbbce2f30fdf166e8 Jul 07 14:35:06.345173 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:06.345131 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mj22f" event={"ID":"2c0d0e79-7cf7-4605-9a72-7823389f4e6d","Type":"ContainerStarted","Data":"5499f7b103cff2b94e3801b80c47f0d3499ef3b92627d8cfbbce2f30fdf166e8"} Jul 07 14:35:06.347364 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:06.347333 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" Jul 07 14:35:07.353417 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:07.353330 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mj22f" event={"ID":"2c0d0e79-7cf7-4605-9a72-7823389f4e6d","Type":"ContainerStarted","Data":"211feb6f007997058ffe5a8212d81cb50a2ef96c737fd44716ec4d7cd3066268"} Jul 07 14:35:07.404973 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:07.404923 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mj22f" podStartSLOduration=2.020820416 podStartE2EDuration="3.40490893s" totalImagesPullingTime="1.384088514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:35:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:35:07.404713642 +0000 UTC m=+508.752767196" watchObservedRunningTime="2026-07-07 14:35:07.40490893 +0000 UTC m=+508.752962483" Jul 07 14:35:08.358788 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:08.358759 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mj22f" Jul 07 14:35:17.356359 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:17.356323 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" Jul 07 14:35:19.366170 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:35:19.366139 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mj22f" Jul 07 14:36:39.066193 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:36:39.066152 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:36:39.067024 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:36:39.067008 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:37:32.235243 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.235214 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7"] Jul 07 14:37:32.238445 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.238430 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.240889 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.240865 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 14:37:32.241031 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.240938 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 07 14:37:32.241236 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.241221 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 14:37:32.241655 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.241636 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wvkhz\"" Jul 07 14:37:32.250756 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.250734 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7"] Jul 07 14:37:32.358671 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.358640 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.358828 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.358695 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.358828 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.358721 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.358828 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.358762 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.358828 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.358783 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.359026 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.358851 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78hk2\" (UniqueName: \"kubernetes.io/projected/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-kube-api-access-78hk2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.459845 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.459809 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78hk2\" (UniqueName: \"kubernetes.io/projected/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-kube-api-access-78hk2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.460033 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.459851 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.460033 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.459887 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.460033 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.459904 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.460033 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.459927 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.460033 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.459942 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.460347 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.460327 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.460396 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.460345 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.460439 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.460412 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.462493 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.462474 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.462660 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.462644 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.468392 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.468372 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78hk2\" (UniqueName: \"kubernetes.io/projected/0e2ab784-7f67-4e0c-bb1d-e433bd9133cb-kube-api-access-78hk2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj6n7\" (UID: \"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.549220 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.549146 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:32.682880 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:32.682856 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7"] Jul 07 14:37:32.685153 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:37:32.685124 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e2ab784_7f67_4e0c_bb1d_e433bd9133cb.slice/crio-94113e66255ea10024fc44b74a800219f20a27c58d20e395b2c1ae00a71eba39 WatchSource:0}: Error finding container 94113e66255ea10024fc44b74a800219f20a27c58d20e395b2c1ae00a71eba39: Status 404 returned error can't find the container with id 94113e66255ea10024fc44b74a800219f20a27c58d20e395b2c1ae00a71eba39 Jul 07 14:37:33.012912 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:33.012874 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" event={"ID":"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb","Type":"ContainerStarted","Data":"94113e66255ea10024fc44b74a800219f20a27c58d20e395b2c1ae00a71eba39"} Jul 07 14:37:34.020088 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:34.019976 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" event={"ID":"0e2ab784-7f67-4e0c-bb1d-e433bd9133cb","Type":"ContainerStarted","Data":"6d547dcf7483ae1ec3a6cf70a254bfb8b94540b4e65bf5971735deeaaa28b048"} Jul 07 14:37:34.039614 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:34.039568 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" podStartSLOduration=0.96902082 podStartE2EDuration="2.039552352s" totalImagesPullingTime="1.070531532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:37:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:37:34.037462482 +0000 UTC m=+655.385516046" watchObservedRunningTime="2026-07-07 14:37:34.039552352 +0000 UTC m=+655.387605905" Jul 07 14:37:42.549935 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:42.549877 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:42.550369 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:42.549965 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:42.562400 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:42.562380 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:37:43.077260 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:37:43.077231 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj6n7" Jul 07 14:38:04.537851 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.537813 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7"] Jul 07 14:38:04.540305 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.540288 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.542376 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.542359 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 07 14:38:04.552703 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.552680 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7"] Jul 07 14:38:04.650729 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.650699 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86b5da0d-c202-46e1-a258-87b8ef53002c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.650729 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.650733 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86b5da0d-c202-46e1-a258-87b8ef53002c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.650914 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.650768 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86b5da0d-c202-46e1-a258-87b8ef53002c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.650914 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.650854 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86b5da0d-c202-46e1-a258-87b8ef53002c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.650914 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.650871 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl28k\" (UniqueName: \"kubernetes.io/projected/86b5da0d-c202-46e1-a258-87b8ef53002c-kube-api-access-hl28k\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.650914 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.650889 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86b5da0d-c202-46e1-a258-87b8ef53002c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.751782 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.751754 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86b5da0d-c202-46e1-a258-87b8ef53002c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.751782 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.751785 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86b5da0d-c202-46e1-a258-87b8ef53002c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.752015 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.751817 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86b5da0d-c202-46e1-a258-87b8ef53002c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.752015 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.751946 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86b5da0d-c202-46e1-a258-87b8ef53002c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.752015 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.751979 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl28k\" (UniqueName: \"kubernetes.io/projected/86b5da0d-c202-46e1-a258-87b8ef53002c-kube-api-access-hl28k\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.752015 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.752013 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86b5da0d-c202-46e1-a258-87b8ef53002c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.752355 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.752321 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86b5da0d-c202-46e1-a258-87b8ef53002c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.752355 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.752341 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86b5da0d-c202-46e1-a258-87b8ef53002c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.752707 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.752369 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86b5da0d-c202-46e1-a258-87b8ef53002c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.754217 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.754195 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86b5da0d-c202-46e1-a258-87b8ef53002c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.754515 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.754498 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86b5da0d-c202-46e1-a258-87b8ef53002c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.760735 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.760715 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl28k\" (UniqueName: \"kubernetes.io/projected/86b5da0d-c202-46e1-a258-87b8ef53002c-kube-api-access-hl28k\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7\" (UID: \"86b5da0d-c202-46e1-a258-87b8ef53002c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.850556 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.850462 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:04.985833 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.985801 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7"] Jul 07 14:38:04.991742 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:04.991694 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 14:38:05.166296 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:05.166218 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" event={"ID":"86b5da0d-c202-46e1-a258-87b8ef53002c","Type":"ContainerStarted","Data":"0740c97af6148d9a414fcb55ff820390009a5713dd027dc66d778168f62ac565"} Jul 07 14:38:06.172058 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:06.172021 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" event={"ID":"86b5da0d-c202-46e1-a258-87b8ef53002c","Type":"ContainerStarted","Data":"20d970be00541746e32aea4a2dff3eb689819b38f4c7dbc4880bf360874cafe1"} Jul 07 14:38:06.190908 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:06.190861 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" podStartSLOduration=1.8737382089999999 podStartE2EDuration="2.190845625s" totalImagesPullingTime="317.107416ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:38:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:38:06.189155074 +0000 UTC m=+687.537208628" watchObservedRunningTime="2026-07-07 14:38:06.190845625 +0000 UTC m=+687.538899177" Jul 07 14:38:14.850810 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:14.850778 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:14.851322 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:14.850837 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:14.863523 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:14.863498 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:38:15.228729 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:38:15.228700 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7" Jul 07 14:41:39.095483 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:41:39.095455 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:41:39.096753 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:41:39.096733 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:46:12.308746 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:12.308658 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29"] Jul 07 14:46:12.309262 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:12.308913 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" podUID="6a1e446f-7ce9-4f73-9c9a-9fb573e428f9" containerName="manager" containerID="cri-o://2ebd3a641edb3b07a2cdcaa7e243e507f4d9e641142724e2b6cbac0c6da0f573" gracePeriod=10 Jul 07 14:46:15.353156 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.353130 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" Jul 07 14:46:15.444224 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.444137 2555 generic.go:356] "Generic (PLEG): container finished" podID="6a1e446f-7ce9-4f73-9c9a-9fb573e428f9" containerID="2ebd3a641edb3b07a2cdcaa7e243e507f4d9e641142724e2b6cbac0c6da0f573" exitCode=0 Jul 07 14:46:15.444224 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.444184 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" Jul 07 14:46:15.444224 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.444191 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" event={"ID":"6a1e446f-7ce9-4f73-9c9a-9fb573e428f9","Type":"ContainerDied","Data":"2ebd3a641edb3b07a2cdcaa7e243e507f4d9e641142724e2b6cbac0c6da0f573"} Jul 07 14:46:15.444482 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.444232 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29" event={"ID":"6a1e446f-7ce9-4f73-9c9a-9fb573e428f9","Type":"ContainerDied","Data":"d639761478d1612435fbeb93f814ab630359c6c59247fb2a5120f9ed3e8bb010"} Jul 07 14:46:15.444482 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.444255 2555 scope.go:117] "RemoveContainer" containerID="2ebd3a641edb3b07a2cdcaa7e243e507f4d9e641142724e2b6cbac0c6da0f573" Jul 07 14:46:15.445943 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.445919 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkp2t\" (UniqueName: \"kubernetes.io/projected/6a1e446f-7ce9-4f73-9c9a-9fb573e428f9-kube-api-access-wkp2t\") pod \"6a1e446f-7ce9-4f73-9c9a-9fb573e428f9\" (UID: \"6a1e446f-7ce9-4f73-9c9a-9fb573e428f9\") " Jul 07 14:46:15.446093 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.445978 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a1e446f-7ce9-4f73-9c9a-9fb573e428f9-extensions-socket-volume\") pod \"6a1e446f-7ce9-4f73-9c9a-9fb573e428f9\" (UID: \"6a1e446f-7ce9-4f73-9c9a-9fb573e428f9\") " Jul 07 14:46:15.446447 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.446421 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1e446f-7ce9-4f73-9c9a-9fb573e428f9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6a1e446f-7ce9-4f73-9c9a-9fb573e428f9" (UID: "6a1e446f-7ce9-4f73-9c9a-9fb573e428f9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:46:15.448145 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.448122 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a1e446f-7ce9-4f73-9c9a-9fb573e428f9-kube-api-access-wkp2t" (OuterVolumeSpecName: "kube-api-access-wkp2t") pod "6a1e446f-7ce9-4f73-9c9a-9fb573e428f9" (UID: "6a1e446f-7ce9-4f73-9c9a-9fb573e428f9"). InnerVolumeSpecName "kube-api-access-wkp2t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:46:15.461918 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.461901 2555 scope.go:117] "RemoveContainer" containerID="2ebd3a641edb3b07a2cdcaa7e243e507f4d9e641142724e2b6cbac0c6da0f573" Jul 07 14:46:15.462200 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:46:15.462180 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ebd3a641edb3b07a2cdcaa7e243e507f4d9e641142724e2b6cbac0c6da0f573\": container with ID starting with 2ebd3a641edb3b07a2cdcaa7e243e507f4d9e641142724e2b6cbac0c6da0f573 not found: ID does not exist" containerID="2ebd3a641edb3b07a2cdcaa7e243e507f4d9e641142724e2b6cbac0c6da0f573" Jul 07 14:46:15.462251 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.462208 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ebd3a641edb3b07a2cdcaa7e243e507f4d9e641142724e2b6cbac0c6da0f573"} err="failed to get container status \"2ebd3a641edb3b07a2cdcaa7e243e507f4d9e641142724e2b6cbac0c6da0f573\": rpc error: code = NotFound desc = could not find container \"2ebd3a641edb3b07a2cdcaa7e243e507f4d9e641142724e2b6cbac0c6da0f573\": container with ID starting with 2ebd3a641edb3b07a2cdcaa7e243e507f4d9e641142724e2b6cbac0c6da0f573 not found: ID does not exist" Jul 07 14:46:15.547022 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.546987 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkp2t\" (UniqueName: \"kubernetes.io/projected/6a1e446f-7ce9-4f73-9c9a-9fb573e428f9-kube-api-access-wkp2t\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:46:15.547022 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.547018 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a1e446f-7ce9-4f73-9c9a-9fb573e428f9-extensions-socket-volume\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:46:15.766685 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.766661 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29"] Jul 07 14:46:15.771744 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:15.771717 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vrp29"] Jul 07 14:46:17.181922 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:17.181875 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a1e446f-7ce9-4f73-9c9a-9fb573e428f9" path="/var/lib/kubelet/pods/6a1e446f-7ce9-4f73-9c9a-9fb573e428f9/volumes" Jul 07 14:46:39.122148 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:39.122118 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:46:39.125108 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:46:39.125061 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:47:13.448178 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:13.448146 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8"] Jul 07 14:47:13.448631 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:13.448494 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a1e446f-7ce9-4f73-9c9a-9fb573e428f9" containerName="manager" Jul 07 14:47:13.448631 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:13.448504 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a1e446f-7ce9-4f73-9c9a-9fb573e428f9" containerName="manager" Jul 07 14:47:13.448631 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:13.448565 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a1e446f-7ce9-4f73-9c9a-9fb573e428f9" containerName="manager" Jul 07 14:47:13.450566 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:13.450551 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8" Jul 07 14:47:13.457411 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:13.457395 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f2kgf\"" Jul 07 14:47:13.489569 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:13.489540 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8"] Jul 07 14:47:13.581439 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:13.581398 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/657af0d6-c76a-414c-a1bf-3a2546643bab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fpwq8\" (UID: \"657af0d6-c76a-414c-a1bf-3a2546643bab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8" Jul 07 14:47:13.581607 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:13.581458 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqnj7\" (UniqueName: \"kubernetes.io/projected/657af0d6-c76a-414c-a1bf-3a2546643bab-kube-api-access-sqnj7\") pod \"kuadrant-operator-controller-manager-86878c769f-fpwq8\" (UID: \"657af0d6-c76a-414c-a1bf-3a2546643bab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8" Jul 07 14:47:13.682001 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:13.681965 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqnj7\" (UniqueName: \"kubernetes.io/projected/657af0d6-c76a-414c-a1bf-3a2546643bab-kube-api-access-sqnj7\") pod \"kuadrant-operator-controller-manager-86878c769f-fpwq8\" (UID: \"657af0d6-c76a-414c-a1bf-3a2546643bab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8" Jul 07 14:47:13.682195 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:13.682051 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/657af0d6-c76a-414c-a1bf-3a2546643bab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fpwq8\" (UID: \"657af0d6-c76a-414c-a1bf-3a2546643bab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8" Jul 07 14:47:13.682395 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:13.682378 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/657af0d6-c76a-414c-a1bf-3a2546643bab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fpwq8\" (UID: \"657af0d6-c76a-414c-a1bf-3a2546643bab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8" Jul 07 14:47:13.694767 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:13.694737 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqnj7\" (UniqueName: \"kubernetes.io/projected/657af0d6-c76a-414c-a1bf-3a2546643bab-kube-api-access-sqnj7\") pod \"kuadrant-operator-controller-manager-86878c769f-fpwq8\" (UID: \"657af0d6-c76a-414c-a1bf-3a2546643bab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8" Jul 07 14:47:13.760646 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:13.760614 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8" Jul 07 14:47:13.893431 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:13.893396 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8"] Jul 07 14:47:13.896115 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:47:13.896090 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod657af0d6_c76a_414c_a1bf_3a2546643bab.slice/crio-218d5e8e543de42596d7a9dbe8aa71676b052b9d233d9dbf384f5d0b9be5e33f WatchSource:0}: Error finding container 218d5e8e543de42596d7a9dbe8aa71676b052b9d233d9dbf384f5d0b9be5e33f: Status 404 returned error can't find the container with id 218d5e8e543de42596d7a9dbe8aa71676b052b9d233d9dbf384f5d0b9be5e33f Jul 07 14:47:14.711353 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:14.711318 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8" event={"ID":"657af0d6-c76a-414c-a1bf-3a2546643bab","Type":"ContainerStarted","Data":"b7c4557ff0df6679fa10e26bb1ca07a669049c9e429e9532bca8807f6585c59e"} Jul 07 14:47:14.711844 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:14.711376 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8" event={"ID":"657af0d6-c76a-414c-a1bf-3a2546643bab","Type":"ContainerStarted","Data":"218d5e8e543de42596d7a9dbe8aa71676b052b9d233d9dbf384f5d0b9be5e33f"} Jul 07 14:47:14.711844 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:14.711427 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8" Jul 07 14:47:14.731735 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:14.731684 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8" podStartSLOduration=1.731670919 podStartE2EDuration="1.731670919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:47:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:47:14.729878974 +0000 UTC m=+1236.077932528" watchObservedRunningTime="2026-07-07 14:47:14.731670919 +0000 UTC m=+1236.079724535" Jul 07 14:47:25.718259 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:47:25.718233 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpwq8" Jul 07 14:51:39.151160 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:51:39.151129 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:51:39.154442 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:51:39.154416 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:53:38.426938 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.426866 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4"] Jul 07 14:53:38.430377 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.430344 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.434267 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.434250 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9e0aa07d-openshift-default-dockercfg-nt9tr\"" Jul 07 14:53:38.498339 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.498305 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4"] Jul 07 14:53:38.567309 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.567275 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-workload-socket\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.567309 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.567309 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-data\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.567551 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.567326 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-token\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.567551 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.567472 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-envoy\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.567551 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.567507 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5l4p\" (UniqueName: \"kubernetes.io/projected/ab5e3231-c099-4370-914b-4fc8a6e23841-kube-api-access-z5l4p\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.567676 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.567577 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-credential-socket\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.567676 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.567613 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-workload-certs\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.567676 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.567630 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab5e3231-c099-4370-914b-4fc8a6e23841-istiod-ca-cert\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.567676 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.567656 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-podinfo\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.668195 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.668159 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-podinfo\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.668385 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.668209 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-workload-socket\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.668385 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.668224 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-data\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.668385 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.668242 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-token\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.668385 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.668295 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-envoy\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.668385 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.668311 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5l4p\" (UniqueName: \"kubernetes.io/projected/ab5e3231-c099-4370-914b-4fc8a6e23841-kube-api-access-z5l4p\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.668385 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.668340 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-credential-socket\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.668385 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.668368 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-workload-certs\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.668717 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.668391 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab5e3231-c099-4370-914b-4fc8a6e23841-istiod-ca-cert\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.668717 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.668648 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-workload-socket\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.668849 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.668823 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-credential-socket\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.668905 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.668874 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-workload-certs\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.668997 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.668979 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-data\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.669094 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.669048 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab5e3231-c099-4370-914b-4fc8a6e23841-istiod-ca-cert\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.670899 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.670875 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-envoy\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.670983 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.670969 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-podinfo\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.676510 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.676486 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-token\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.677428 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.677378 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5l4p\" (UniqueName: \"kubernetes.io/projected/ab5e3231-c099-4370-914b-4fc8a6e23841-kube-api-access-z5l4p\") pod \"e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.741078 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.741034 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:38.878677 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.878645 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4"] Jul 07 14:53:38.881000 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:53:38.880967 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab5e3231_c099_4370_914b_4fc8a6e23841.slice/crio-6ccbefddf8d6d66fac55e219f2c2bf56d17ef1efa7dc4f7ce0c5482284a9de89 WatchSource:0}: Error finding container 6ccbefddf8d6d66fac55e219f2c2bf56d17ef1efa7dc4f7ce0c5482284a9de89: Status 404 returned error can't find the container with id 6ccbefddf8d6d66fac55e219f2c2bf56d17ef1efa7dc4f7ce0c5482284a9de89 Jul 07 14:53:38.883224 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.883190 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:53:38.883334 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.883256 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:53:38.883334 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:38.883284 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:53:39.510208 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:39.510175 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" event={"ID":"ab5e3231-c099-4370-914b-4fc8a6e23841","Type":"ContainerStarted","Data":"7ffd7a2113d938ea8c7bf86f1aca857534802e02a02eed173aa50c7664389034"} Jul 07 14:53:39.510587 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:39.510226 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" event={"ID":"ab5e3231-c099-4370-914b-4fc8a6e23841","Type":"ContainerStarted","Data":"6ccbefddf8d6d66fac55e219f2c2bf56d17ef1efa7dc4f7ce0c5482284a9de89"} Jul 07 14:53:39.535874 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:39.535820 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" podStartSLOduration=1.535802933 podStartE2EDuration="1.535802933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:53:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:53:39.533883011 +0000 UTC m=+1620.881936567" watchObservedRunningTime="2026-07-07 14:53:39.535802933 +0000 UTC m=+1620.883856487" Jul 07 14:53:39.742270 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:39.742232 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:39.747336 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:39.747309 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:40.514966 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:40.514938 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:40.515925 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:40.515907 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:44.991189 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:44.991152 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4"] Jul 07 14:53:44.991576 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:44.991354 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" podUID="ab5e3231-c099-4370-914b-4fc8a6e23841" containerName="istio-proxy" containerID="cri-o://7ffd7a2113d938ea8c7bf86f1aca857534802e02a02eed173aa50c7664389034" gracePeriod=30 Jul 07 14:53:50.234410 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.234383 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:50.377999 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.377911 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-workload-socket\") pod \"ab5e3231-c099-4370-914b-4fc8a6e23841\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " Jul 07 14:53:50.377999 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.377953 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-workload-certs\") pod \"ab5e3231-c099-4370-914b-4fc8a6e23841\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " Jul 07 14:53:50.378248 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.378000 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5l4p\" (UniqueName: \"kubernetes.io/projected/ab5e3231-c099-4370-914b-4fc8a6e23841-kube-api-access-z5l4p\") pod \"ab5e3231-c099-4370-914b-4fc8a6e23841\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " Jul 07 14:53:50.378248 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.378027 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-envoy\") pod \"ab5e3231-c099-4370-914b-4fc8a6e23841\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " Jul 07 14:53:50.378248 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.378056 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-token\") pod \"ab5e3231-c099-4370-914b-4fc8a6e23841\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " Jul 07 14:53:50.378248 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.378098 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-credential-socket\") pod \"ab5e3231-c099-4370-914b-4fc8a6e23841\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " Jul 07 14:53:50.378248 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.378140 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-podinfo\") pod \"ab5e3231-c099-4370-914b-4fc8a6e23841\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " Jul 07 14:53:50.378248 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.378181 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab5e3231-c099-4370-914b-4fc8a6e23841-istiod-ca-cert\") pod \"ab5e3231-c099-4370-914b-4fc8a6e23841\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " Jul 07 14:53:50.378248 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.378204 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab5e3231-c099-4370-914b-4fc8a6e23841" (UID: "ab5e3231-c099-4370-914b-4fc8a6e23841"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:53:50.378248 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.378244 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-data\") pod \"ab5e3231-c099-4370-914b-4fc8a6e23841\" (UID: \"ab5e3231-c099-4370-914b-4fc8a6e23841\") " Jul 07 14:53:50.378629 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.378385 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab5e3231-c099-4370-914b-4fc8a6e23841" (UID: "ab5e3231-c099-4370-914b-4fc8a6e23841"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:53:50.378629 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.378418 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab5e3231-c099-4370-914b-4fc8a6e23841" (UID: "ab5e3231-c099-4370-914b-4fc8a6e23841"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:53:50.378629 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.378579 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab5e3231-c099-4370-914b-4fc8a6e23841-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab5e3231-c099-4370-914b-4fc8a6e23841" (UID: "ab5e3231-c099-4370-914b-4fc8a6e23841"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:53:50.378629 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.378584 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab5e3231-c099-4370-914b-4fc8a6e23841" (UID: "ab5e3231-c099-4370-914b-4fc8a6e23841"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:53:50.378791 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.378643 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-workload-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:53:50.378791 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.378663 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-workload-certs\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:53:50.378791 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.378679 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-credential-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:53:50.380333 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.380301 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab5e3231-c099-4370-914b-4fc8a6e23841-kube-api-access-z5l4p" (OuterVolumeSpecName: "kube-api-access-z5l4p") pod "ab5e3231-c099-4370-914b-4fc8a6e23841" (UID: "ab5e3231-c099-4370-914b-4fc8a6e23841"). InnerVolumeSpecName "kube-api-access-z5l4p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:53:50.380541 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.380524 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab5e3231-c099-4370-914b-4fc8a6e23841" (UID: "ab5e3231-c099-4370-914b-4fc8a6e23841"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 14:53:50.380715 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.380690 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab5e3231-c099-4370-914b-4fc8a6e23841" (UID: "ab5e3231-c099-4370-914b-4fc8a6e23841"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:53:50.380715 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.380704 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab5e3231-c099-4370-914b-4fc8a6e23841" (UID: "ab5e3231-c099-4370-914b-4fc8a6e23841"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:53:50.479844 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.479811 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5l4p\" (UniqueName: \"kubernetes.io/projected/ab5e3231-c099-4370-914b-4fc8a6e23841-kube-api-access-z5l4p\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:53:50.479844 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.479841 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-envoy\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:53:50.479844 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.479852 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-token\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:53:50.480064 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.479866 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-podinfo\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:53:50.480064 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.479876 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab5e3231-c099-4370-914b-4fc8a6e23841-istiod-ca-cert\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:53:50.480064 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.479884 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab5e3231-c099-4370-914b-4fc8a6e23841-istio-data\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:53:50.562586 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.562557 2555 generic.go:356] "Generic (PLEG): container finished" podID="ab5e3231-c099-4370-914b-4fc8a6e23841" containerID="7ffd7a2113d938ea8c7bf86f1aca857534802e02a02eed173aa50c7664389034" exitCode=0 Jul 07 14:53:50.562740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.562605 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" Jul 07 14:53:50.562740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.562648 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" event={"ID":"ab5e3231-c099-4370-914b-4fc8a6e23841","Type":"ContainerDied","Data":"7ffd7a2113d938ea8c7bf86f1aca857534802e02a02eed173aa50c7664389034"} Jul 07 14:53:50.562740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.562706 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4" event={"ID":"ab5e3231-c099-4370-914b-4fc8a6e23841","Type":"ContainerDied","Data":"6ccbefddf8d6d66fac55e219f2c2bf56d17ef1efa7dc4f7ce0c5482284a9de89"} Jul 07 14:53:50.562740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.562736 2555 scope.go:117] "RemoveContainer" containerID="7ffd7a2113d938ea8c7bf86f1aca857534802e02a02eed173aa50c7664389034" Jul 07 14:53:50.572241 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.572220 2555 scope.go:117] "RemoveContainer" containerID="7ffd7a2113d938ea8c7bf86f1aca857534802e02a02eed173aa50c7664389034" Jul 07 14:53:50.572479 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:53:50.572461 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ffd7a2113d938ea8c7bf86f1aca857534802e02a02eed173aa50c7664389034\": container with ID starting with 7ffd7a2113d938ea8c7bf86f1aca857534802e02a02eed173aa50c7664389034 not found: ID does not exist" containerID="7ffd7a2113d938ea8c7bf86f1aca857534802e02a02eed173aa50c7664389034" Jul 07 14:53:50.572529 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.572486 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ffd7a2113d938ea8c7bf86f1aca857534802e02a02eed173aa50c7664389034"} err="failed to get container status \"7ffd7a2113d938ea8c7bf86f1aca857534802e02a02eed173aa50c7664389034\": rpc error: code = NotFound desc = could not find container \"7ffd7a2113d938ea8c7bf86f1aca857534802e02a02eed173aa50c7664389034\": container with ID starting with 7ffd7a2113d938ea8c7bf86f1aca857534802e02a02eed173aa50c7664389034 not found: ID does not exist" Jul 07 14:53:50.584056 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.584023 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4"] Jul 07 14:53:50.588019 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:50.588001 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9e0aa07d-openshift-default-c4bfc8489-gs4s4"] Jul 07 14:53:51.180474 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.180437 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab5e3231-c099-4370-914b-4fc8a6e23841" path="/var/lib/kubelet/pods/ab5e3231-c099-4370-914b-4fc8a6e23841/volumes" Jul 07 14:53:51.692888 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.692859 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6"] Jul 07 14:53:51.693309 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.693245 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab5e3231-c099-4370-914b-4fc8a6e23841" containerName="istio-proxy" Jul 07 14:53:51.693309 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.693259 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab5e3231-c099-4370-914b-4fc8a6e23841" containerName="istio-proxy" Jul 07 14:53:51.693390 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.693316 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab5e3231-c099-4370-914b-4fc8a6e23841" containerName="istio-proxy" Jul 07 14:53:51.697851 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.697822 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.700841 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.700820 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4198bef9-openshift-default-dockercfg-2pwvv\"" Jul 07 14:53:51.712696 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.712672 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6"] Jul 07 14:53:51.792279 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.792240 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-istio-data\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.792438 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.792285 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115546cb-afba-461b-949e-aa41d775c87b-istio-podinfo\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.792438 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.792329 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-workload-certs\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.792438 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.792384 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-istio-envoy\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.792549 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.792445 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115546cb-afba-461b-949e-aa41d775c87b-istio-token\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.792549 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.792517 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115546cb-afba-461b-949e-aa41d775c87b-istiod-ca-cert\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.792610 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.792567 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z68r6\" (UniqueName: \"kubernetes.io/projected/115546cb-afba-461b-949e-aa41d775c87b-kube-api-access-z68r6\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.792645 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.792624 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-workload-socket\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.792678 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.792664 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-credential-socket\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.893919 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.893884 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z68r6\" (UniqueName: \"kubernetes.io/projected/115546cb-afba-461b-949e-aa41d775c87b-kube-api-access-z68r6\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.893919 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.893925 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-workload-socket\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.894141 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.893944 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-credential-socket\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.894141 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.893971 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-istio-data\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.894141 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.893995 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115546cb-afba-461b-949e-aa41d775c87b-istio-podinfo\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.894141 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.894029 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-workload-certs\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.894141 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.894058 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-istio-envoy\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.894141 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.894134 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115546cb-afba-461b-949e-aa41d775c87b-istio-token\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.894443 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.894218 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115546cb-afba-461b-949e-aa41d775c87b-istiod-ca-cert\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.894443 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.894402 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-workload-socket\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.894543 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.894439 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-istio-data\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.894588 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.894542 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-credential-socket\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.894628 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.894612 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-workload-certs\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.894934 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.894918 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115546cb-afba-461b-949e-aa41d775c87b-istiod-ca-cert\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.896520 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.896497 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-istio-envoy\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.896752 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.896729 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115546cb-afba-461b-949e-aa41d775c87b-istio-podinfo\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.901969 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.901940 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115546cb-afba-461b-949e-aa41d775c87b-istio-token\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:51.902203 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:51.902186 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z68r6\" (UniqueName: \"kubernetes.io/projected/115546cb-afba-461b-949e-aa41d775c87b-kube-api-access-z68r6\") pod \"e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:52.007942 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:52.007908 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:52.145214 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:52.145179 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6"] Jul 07 14:53:52.148242 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:53:52.148203 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod115546cb_afba_461b_949e_aa41d775c87b.slice/crio-228de51b8333a3070c9d533c1f31e8978d6eb766f36f1f57f02514fd7dca9804 WatchSource:0}: Error finding container 228de51b8333a3070c9d533c1f31e8978d6eb766f36f1f57f02514fd7dca9804: Status 404 returned error can't find the container with id 228de51b8333a3070c9d533c1f31e8978d6eb766f36f1f57f02514fd7dca9804 Jul 07 14:53:52.150475 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:52.150439 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:53:52.150559 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:52.150502 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:53:52.150559 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:52.150529 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:53:52.575807 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:52.575771 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" event={"ID":"115546cb-afba-461b-949e-aa41d775c87b","Type":"ContainerStarted","Data":"50cf5eb4e122f3709e62079dde5966904c55af1612e37064d5b5b150794c29b3"} Jul 07 14:53:52.575989 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:52.575826 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" event={"ID":"115546cb-afba-461b-949e-aa41d775c87b","Type":"ContainerStarted","Data":"228de51b8333a3070c9d533c1f31e8978d6eb766f36f1f57f02514fd7dca9804"} Jul 07 14:53:52.598532 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:52.598479 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" podStartSLOduration=1.598462461 podStartE2EDuration="1.598462461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:53:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:53:52.59678265 +0000 UTC m=+1633.944836214" watchObservedRunningTime="2026-07-07 14:53:52.598462461 +0000 UTC m=+1633.946516013" Jul 07 14:53:53.008150 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:53.008119 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:53.019839 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:53.019810 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:53.580592 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:53.580560 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:53:53.581569 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:53:53.581551 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:54:03.641568 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:03.641482 2555 generic.go:356] "Generic (PLEG): container finished" podID="115546cb-afba-461b-949e-aa41d775c87b" containerID="50cf5eb4e122f3709e62079dde5966904c55af1612e37064d5b5b150794c29b3" exitCode=0 Jul 07 14:54:03.641965 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:03.641556 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" event={"ID":"115546cb-afba-461b-949e-aa41d775c87b","Type":"ContainerDied","Data":"50cf5eb4e122f3709e62079dde5966904c55af1612e37064d5b5b150794c29b3"} Jul 07 14:54:03.642006 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:03.641987 2555 scope.go:117] "RemoveContainer" containerID="50cf5eb4e122f3709e62079dde5966904c55af1612e37064d5b5b150794c29b3" Jul 07 14:54:03.642960 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:03.642938 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:54:03.643023 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:03.643009 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:54:03.643058 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:03.643042 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:54:04.417398 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:04.417367 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6"] Jul 07 14:54:04.649108 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:04.649048 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" event={"ID":"115546cb-afba-461b-949e-aa41d775c87b","Type":"ContainerStarted","Data":"d385b7aeec09e484912498155f6b78d6aa1559c72837329c06a3a2268ac867ff"} Jul 07 14:54:04.649529 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:04.649162 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" podUID="115546cb-afba-461b-949e-aa41d775c87b" containerName="istio-proxy" containerID="cri-o://d385b7aeec09e484912498155f6b78d6aa1559c72837329c06a3a2268ac867ff" gracePeriod=30 Jul 07 14:54:08.581444 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:08.581408 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:54:09.903401 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.903367 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:54:09.963385 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.963300 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115546cb-afba-461b-949e-aa41d775c87b-istio-podinfo\") pod \"115546cb-afba-461b-949e-aa41d775c87b\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " Jul 07 14:54:09.963385 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.963337 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-workload-certs\") pod \"115546cb-afba-461b-949e-aa41d775c87b\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " Jul 07 14:54:09.963385 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.963367 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-istio-data\") pod \"115546cb-afba-461b-949e-aa41d775c87b\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " Jul 07 14:54:09.963637 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.963405 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-workload-socket\") pod \"115546cb-afba-461b-949e-aa41d775c87b\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " Jul 07 14:54:09.963637 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.963500 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-credential-socket\") pod \"115546cb-afba-461b-949e-aa41d775c87b\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " Jul 07 14:54:09.963637 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.963555 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115546cb-afba-461b-949e-aa41d775c87b-istio-token\") pod \"115546cb-afba-461b-949e-aa41d775c87b\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " Jul 07 14:54:09.963637 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.963590 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-istio-envoy\") pod \"115546cb-afba-461b-949e-aa41d775c87b\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " Jul 07 14:54:09.963637 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.963614 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115546cb-afba-461b-949e-aa41d775c87b-istiod-ca-cert\") pod \"115546cb-afba-461b-949e-aa41d775c87b\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " Jul 07 14:54:09.963875 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.963664 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z68r6\" (UniqueName: \"kubernetes.io/projected/115546cb-afba-461b-949e-aa41d775c87b-kube-api-access-z68r6\") pod \"115546cb-afba-461b-949e-aa41d775c87b\" (UID: \"115546cb-afba-461b-949e-aa41d775c87b\") " Jul 07 14:54:09.963875 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.963652 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "115546cb-afba-461b-949e-aa41d775c87b" (UID: "115546cb-afba-461b-949e-aa41d775c87b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:54:09.963875 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.963702 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "115546cb-afba-461b-949e-aa41d775c87b" (UID: "115546cb-afba-461b-949e-aa41d775c87b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:54:09.963875 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.963758 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "115546cb-afba-461b-949e-aa41d775c87b" (UID: "115546cb-afba-461b-949e-aa41d775c87b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:54:09.964099 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.963940 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-credential-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:09.964099 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.963959 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-workload-certs\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:09.964099 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.963978 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-workload-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:09.964099 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.964013 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/115546cb-afba-461b-949e-aa41d775c87b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "115546cb-afba-461b-949e-aa41d775c87b" (UID: "115546cb-afba-461b-949e-aa41d775c87b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:54:09.964456 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.964432 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-istio-data" (OuterVolumeSpecName: "istio-data") pod "115546cb-afba-461b-949e-aa41d775c87b" (UID: "115546cb-afba-461b-949e-aa41d775c87b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:54:09.965631 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.965611 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/115546cb-afba-461b-949e-aa41d775c87b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "115546cb-afba-461b-949e-aa41d775c87b" (UID: "115546cb-afba-461b-949e-aa41d775c87b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 14:54:09.965982 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.965961 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "115546cb-afba-461b-949e-aa41d775c87b" (UID: "115546cb-afba-461b-949e-aa41d775c87b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:54:09.966302 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.966279 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/115546cb-afba-461b-949e-aa41d775c87b-istio-token" (OuterVolumeSpecName: "istio-token") pod "115546cb-afba-461b-949e-aa41d775c87b" (UID: "115546cb-afba-461b-949e-aa41d775c87b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:54:09.966400 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:09.966311 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/115546cb-afba-461b-949e-aa41d775c87b-kube-api-access-z68r6" (OuterVolumeSpecName: "kube-api-access-z68r6") pod "115546cb-afba-461b-949e-aa41d775c87b" (UID: "115546cb-afba-461b-949e-aa41d775c87b"). InnerVolumeSpecName "kube-api-access-z68r6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:54:10.064653 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.064622 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115546cb-afba-461b-949e-aa41d775c87b-istio-token\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:10.064653 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.064648 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-istio-envoy\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:10.064653 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.064657 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115546cb-afba-461b-949e-aa41d775c87b-istiod-ca-cert\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:10.064868 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.064666 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z68r6\" (UniqueName: \"kubernetes.io/projected/115546cb-afba-461b-949e-aa41d775c87b-kube-api-access-z68r6\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:10.064868 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.064675 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115546cb-afba-461b-949e-aa41d775c87b-istio-podinfo\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:10.064868 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.064682 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115546cb-afba-461b-949e-aa41d775c87b-istio-data\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:10.678913 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.678875 2555 generic.go:356] "Generic (PLEG): container finished" podID="115546cb-afba-461b-949e-aa41d775c87b" containerID="d385b7aeec09e484912498155f6b78d6aa1559c72837329c06a3a2268ac867ff" exitCode=0 Jul 07 14:54:10.679147 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.678934 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" Jul 07 14:54:10.679147 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.678970 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" event={"ID":"115546cb-afba-461b-949e-aa41d775c87b","Type":"ContainerDied","Data":"d385b7aeec09e484912498155f6b78d6aa1559c72837329c06a3a2268ac867ff"} Jul 07 14:54:10.679147 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.679039 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6" event={"ID":"115546cb-afba-461b-949e-aa41d775c87b","Type":"ContainerDied","Data":"228de51b8333a3070c9d533c1f31e8978d6eb766f36f1f57f02514fd7dca9804"} Jul 07 14:54:10.679147 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.679086 2555 scope.go:117] "RemoveContainer" containerID="d385b7aeec09e484912498155f6b78d6aa1559c72837329c06a3a2268ac867ff" Jul 07 14:54:10.689205 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.689147 2555 scope.go:117] "RemoveContainer" containerID="50cf5eb4e122f3709e62079dde5966904c55af1612e37064d5b5b150794c29b3" Jul 07 14:54:10.697519 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.697504 2555 scope.go:117] "RemoveContainer" containerID="d385b7aeec09e484912498155f6b78d6aa1559c72837329c06a3a2268ac867ff" Jul 07 14:54:10.697824 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:54:10.697804 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d385b7aeec09e484912498155f6b78d6aa1559c72837329c06a3a2268ac867ff\": container with ID starting with d385b7aeec09e484912498155f6b78d6aa1559c72837329c06a3a2268ac867ff not found: ID does not exist" containerID="d385b7aeec09e484912498155f6b78d6aa1559c72837329c06a3a2268ac867ff" Jul 07 14:54:10.697894 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.697841 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d385b7aeec09e484912498155f6b78d6aa1559c72837329c06a3a2268ac867ff"} err="failed to get container status \"d385b7aeec09e484912498155f6b78d6aa1559c72837329c06a3a2268ac867ff\": rpc error: code = NotFound desc = could not find container \"d385b7aeec09e484912498155f6b78d6aa1559c72837329c06a3a2268ac867ff\": container with ID starting with d385b7aeec09e484912498155f6b78d6aa1559c72837329c06a3a2268ac867ff not found: ID does not exist" Jul 07 14:54:10.697894 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.697865 2555 scope.go:117] "RemoveContainer" containerID="50cf5eb4e122f3709e62079dde5966904c55af1612e37064d5b5b150794c29b3" Jul 07 14:54:10.698129 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:54:10.698109 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50cf5eb4e122f3709e62079dde5966904c55af1612e37064d5b5b150794c29b3\": container with ID starting with 50cf5eb4e122f3709e62079dde5966904c55af1612e37064d5b5b150794c29b3 not found: ID does not exist" containerID="50cf5eb4e122f3709e62079dde5966904c55af1612e37064d5b5b150794c29b3" Jul 07 14:54:10.698173 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.698137 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50cf5eb4e122f3709e62079dde5966904c55af1612e37064d5b5b150794c29b3"} err="failed to get container status \"50cf5eb4e122f3709e62079dde5966904c55af1612e37064d5b5b150794c29b3\": rpc error: code = NotFound desc = could not find container \"50cf5eb4e122f3709e62079dde5966904c55af1612e37064d5b5b150794c29b3\": container with ID starting with 50cf5eb4e122f3709e62079dde5966904c55af1612e37064d5b5b150794c29b3 not found: ID does not exist" Jul 07 14:54:10.703963 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.703936 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6"] Jul 07 14:54:10.707735 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:10.707714 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4198bef9-openshift-default-6cb8898dd7-pfkr6"] Jul 07 14:54:11.180662 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.180623 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="115546cb-afba-461b-949e-aa41d775c87b" path="/var/lib/kubelet/pods/115546cb-afba-461b-949e-aa41d775c87b/volumes" Jul 07 14:54:11.242962 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.242933 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5"] Jul 07 14:54:11.243401 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.243382 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="115546cb-afba-461b-949e-aa41d775c87b" containerName="istio-proxy" Jul 07 14:54:11.243501 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.243403 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="115546cb-afba-461b-949e-aa41d775c87b" containerName="istio-proxy" Jul 07 14:54:11.243501 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.243425 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="115546cb-afba-461b-949e-aa41d775c87b" containerName="istio-proxy" Jul 07 14:54:11.243501 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.243434 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="115546cb-afba-461b-949e-aa41d775c87b" containerName="istio-proxy" Jul 07 14:54:11.243647 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.243539 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="115546cb-afba-461b-949e-aa41d775c87b" containerName="istio-proxy" Jul 07 14:54:11.246801 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.246781 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.249368 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.249346 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-35928690-openshift-default-dockercfg-4tvrb\"" Jul 07 14:54:11.263572 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.263551 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5"] Jul 07 14:54:11.376214 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.376181 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-envoy\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.376428 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.376237 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w8sn\" (UniqueName: \"kubernetes.io/projected/1210ab41-0c53-461a-8ecf-0457de3b4027-kube-api-access-8w8sn\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.376428 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.376344 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-data\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.376537 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.376467 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-credential-socket\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.376537 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.376520 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-workload-socket\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.376624 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.376548 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-podinfo\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.376624 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.376580 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-workload-certs\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.376624 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.376609 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-token\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.376761 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.376642 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1210ab41-0c53-461a-8ecf-0457de3b4027-istiod-ca-cert\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.477436 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.477398 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-envoy\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.477599 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.477465 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w8sn\" (UniqueName: \"kubernetes.io/projected/1210ab41-0c53-461a-8ecf-0457de3b4027-kube-api-access-8w8sn\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.477599 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.477513 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-data\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.477599 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.477577 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-credential-socket\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.477781 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.477614 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-workload-socket\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.477781 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.477640 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-podinfo\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.477781 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.477670 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-workload-certs\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.477781 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.477696 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-token\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.477781 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.477731 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1210ab41-0c53-461a-8ecf-0457de3b4027-istiod-ca-cert\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.478034 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.477949 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-data\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.478223 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.478148 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-credential-socket\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.478343 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.478238 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-workload-certs\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.478502 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.478479 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-workload-socket\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.478701 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.478511 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1210ab41-0c53-461a-8ecf-0457de3b4027-istiod-ca-cert\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.480391 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.480360 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-envoy\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.480760 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.480739 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-podinfo\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.486289 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.486268 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-token\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.486526 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.486506 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w8sn\" (UniqueName: \"kubernetes.io/projected/1210ab41-0c53-461a-8ecf-0457de3b4027-kube-api-access-8w8sn\") pod \"e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.557673 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.557634 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:11.694823 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.694797 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5"] Jul 07 14:54:11.696837 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:54:11.696799 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1210ab41_0c53_461a_8ecf_0457de3b4027.slice/crio-db66fd81750627e71dea782760f5240bd5cef6c7445c7a8a7b58476fce2035fd WatchSource:0}: Error finding container db66fd81750627e71dea782760f5240bd5cef6c7445c7a8a7b58476fce2035fd: Status 404 returned error can't find the container with id db66fd81750627e71dea782760f5240bd5cef6c7445c7a8a7b58476fce2035fd Jul 07 14:54:11.698951 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.698918 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:54:11.699048 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.699000 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:54:11.699148 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:11.699050 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:54:12.692190 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:12.692149 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" event={"ID":"1210ab41-0c53-461a-8ecf-0457de3b4027","Type":"ContainerStarted","Data":"71284c4bad577c7e016a9c1c51a50c68e3347094d76f839a0ca679ad8632ccc0"} Jul 07 14:54:12.692622 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:12.692207 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" event={"ID":"1210ab41-0c53-461a-8ecf-0457de3b4027","Type":"ContainerStarted","Data":"db66fd81750627e71dea782760f5240bd5cef6c7445c7a8a7b58476fce2035fd"} Jul 07 14:54:12.716870 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:12.716809 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" podStartSLOduration=1.716793439 podStartE2EDuration="1.716793439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:54:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:54:12.716185653 +0000 UTC m=+1654.064239207" watchObservedRunningTime="2026-07-07 14:54:12.716793439 +0000 UTC m=+1654.064846992" Jul 07 14:54:13.558004 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:13.557976 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:13.562711 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:13.562687 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:13.696731 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:13.696698 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:13.697871 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:13.697840 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:17.212739 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:17.212700 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5"] Jul 07 14:54:17.213250 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:17.212952 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" podUID="1210ab41-0c53-461a-8ecf-0457de3b4027" containerName="istio-proxy" containerID="cri-o://71284c4bad577c7e016a9c1c51a50c68e3347094d76f839a0ca679ad8632ccc0" gracePeriod=30 Jul 07 14:54:23.747319 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.747287 2555 generic.go:356] "Generic (PLEG): container finished" podID="1210ab41-0c53-461a-8ecf-0457de3b4027" containerID="71284c4bad577c7e016a9c1c51a50c68e3347094d76f839a0ca679ad8632ccc0" exitCode=0 Jul 07 14:54:23.747655 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.747361 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" event={"ID":"1210ab41-0c53-461a-8ecf-0457de3b4027","Type":"ContainerDied","Data":"71284c4bad577c7e016a9c1c51a50c68e3347094d76f839a0ca679ad8632ccc0"} Jul 07 14:54:23.766333 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.766278 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:23.900918 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.900879 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-workload-socket\") pod \"1210ab41-0c53-461a-8ecf-0457de3b4027\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " Jul 07 14:54:23.900918 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.900922 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-token\") pod \"1210ab41-0c53-461a-8ecf-0457de3b4027\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " Jul 07 14:54:23.901213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.900950 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1210ab41-0c53-461a-8ecf-0457de3b4027-istiod-ca-cert\") pod \"1210ab41-0c53-461a-8ecf-0457de3b4027\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " Jul 07 14:54:23.901213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.900977 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-credential-socket\") pod \"1210ab41-0c53-461a-8ecf-0457de3b4027\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " Jul 07 14:54:23.901213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.901008 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8w8sn\" (UniqueName: \"kubernetes.io/projected/1210ab41-0c53-461a-8ecf-0457de3b4027-kube-api-access-8w8sn\") pod \"1210ab41-0c53-461a-8ecf-0457de3b4027\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " Jul 07 14:54:23.901213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.901035 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-envoy\") pod \"1210ab41-0c53-461a-8ecf-0457de3b4027\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " Jul 07 14:54:23.901213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.901056 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-data\") pod \"1210ab41-0c53-461a-8ecf-0457de3b4027\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " Jul 07 14:54:23.901213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.901108 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-podinfo\") pod \"1210ab41-0c53-461a-8ecf-0457de3b4027\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " Jul 07 14:54:23.901213 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.901157 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-workload-certs\") pod \"1210ab41-0c53-461a-8ecf-0457de3b4027\" (UID: \"1210ab41-0c53-461a-8ecf-0457de3b4027\") " Jul 07 14:54:23.901573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.901324 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1210ab41-0c53-461a-8ecf-0457de3b4027" (UID: "1210ab41-0c53-461a-8ecf-0457de3b4027"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:54:23.901573 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.901478 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1210ab41-0c53-461a-8ecf-0457de3b4027-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1210ab41-0c53-461a-8ecf-0457de3b4027" (UID: "1210ab41-0c53-461a-8ecf-0457de3b4027"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:54:23.901686 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.901595 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1210ab41-0c53-461a-8ecf-0457de3b4027" (UID: "1210ab41-0c53-461a-8ecf-0457de3b4027"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:54:23.901686 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.901637 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-workload-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:23.901686 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.901651 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1210ab41-0c53-461a-8ecf-0457de3b4027-istiod-ca-cert\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:23.901821 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.901747 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1210ab41-0c53-461a-8ecf-0457de3b4027" (UID: "1210ab41-0c53-461a-8ecf-0457de3b4027"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:54:23.901925 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.901895 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-data" (OuterVolumeSpecName: "istio-data") pod "1210ab41-0c53-461a-8ecf-0457de3b4027" (UID: "1210ab41-0c53-461a-8ecf-0457de3b4027"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:54:23.903421 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.903389 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-token" (OuterVolumeSpecName: "istio-token") pod "1210ab41-0c53-461a-8ecf-0457de3b4027" (UID: "1210ab41-0c53-461a-8ecf-0457de3b4027"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:54:23.903767 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.903730 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1210ab41-0c53-461a-8ecf-0457de3b4027" (UID: "1210ab41-0c53-461a-8ecf-0457de3b4027"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 14:54:23.903885 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.903761 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1210ab41-0c53-461a-8ecf-0457de3b4027-kube-api-access-8w8sn" (OuterVolumeSpecName: "kube-api-access-8w8sn") pod "1210ab41-0c53-461a-8ecf-0457de3b4027" (UID: "1210ab41-0c53-461a-8ecf-0457de3b4027"). InnerVolumeSpecName "kube-api-access-8w8sn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:54:23.903885 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:23.903853 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1210ab41-0c53-461a-8ecf-0457de3b4027" (UID: "1210ab41-0c53-461a-8ecf-0457de3b4027"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:54:24.002514 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:24.002480 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-token\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:24.002514 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:24.002508 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-credential-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:24.002514 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:24.002520 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8w8sn\" (UniqueName: \"kubernetes.io/projected/1210ab41-0c53-461a-8ecf-0457de3b4027-kube-api-access-8w8sn\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:24.002748 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:24.002529 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-envoy\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:24.002748 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:24.002537 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-data\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:24.002748 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:24.002544 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1210ab41-0c53-461a-8ecf-0457de3b4027-istio-podinfo\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:24.002748 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:24.002551 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1210ab41-0c53-461a-8ecf-0457de3b4027-workload-certs\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:54:24.753184 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:24.753151 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" Jul 07 14:54:24.753184 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:24.753159 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5" event={"ID":"1210ab41-0c53-461a-8ecf-0457de3b4027","Type":"ContainerDied","Data":"db66fd81750627e71dea782760f5240bd5cef6c7445c7a8a7b58476fce2035fd"} Jul 07 14:54:24.753670 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:24.753229 2555 scope.go:117] "RemoveContainer" containerID="71284c4bad577c7e016a9c1c51a50c68e3347094d76f839a0ca679ad8632ccc0" Jul 07 14:54:24.781767 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:24.781746 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5"] Jul 07 14:54:24.792545 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:24.792525 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-35928690-openshift-default-6d4b85999f-6lpd5"] Jul 07 14:54:25.179601 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:54:25.179519 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1210ab41-0c53-461a-8ecf-0457de3b4027" path="/var/lib/kubelet/pods/1210ab41-0c53-461a-8ecf-0457de3b4027/volumes" Jul 07 14:56:01.658798 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.658766 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq"] Jul 07 14:56:01.659262 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.659164 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1210ab41-0c53-461a-8ecf-0457de3b4027" containerName="istio-proxy" Jul 07 14:56:01.659262 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.659177 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="1210ab41-0c53-461a-8ecf-0457de3b4027" containerName="istio-proxy" Jul 07 14:56:01.659262 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.659241 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="115546cb-afba-461b-949e-aa41d775c87b" containerName="istio-proxy" Jul 07 14:56:01.659262 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.659248 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="1210ab41-0c53-461a-8ecf-0457de3b4027" containerName="istio-proxy" Jul 07 14:56:01.662147 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.662131 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.665057 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.665034 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8dea50b1-openshift-default-dockercfg-fzb84\"" Jul 07 14:56:01.692752 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.692719 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq"] Jul 07 14:56:01.764526 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.764492 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-istio-data\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.764526 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.764531 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-workload-socket\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.764789 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.764546 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e66c6ff8-af72-46c1-a592-c631555b774f-istio-token\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.764789 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.764564 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq2bk\" (UniqueName: \"kubernetes.io/projected/e66c6ff8-af72-46c1-a592-c631555b774f-kube-api-access-cq2bk\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.764789 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.764663 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e66c6ff8-af72-46c1-a592-c631555b774f-istio-podinfo\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.764789 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.764720 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-credential-socket\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.764789 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.764771 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-istio-envoy\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.764985 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.764812 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-workload-certs\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.764985 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.764837 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e66c6ff8-af72-46c1-a592-c631555b774f-istiod-ca-cert\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.866083 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.866028 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e66c6ff8-af72-46c1-a592-c631555b774f-istio-podinfo\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.866284 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.866099 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-credential-socket\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.866284 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.866141 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-istio-envoy\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.866284 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.866173 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-workload-certs\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.866284 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.866195 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e66c6ff8-af72-46c1-a592-c631555b774f-istiod-ca-cert\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.866284 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.866258 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-istio-data\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.866284 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.866283 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-workload-socket\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.866601 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.866305 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e66c6ff8-af72-46c1-a592-c631555b774f-istio-token\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.866601 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.866335 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq2bk\" (UniqueName: \"kubernetes.io/projected/e66c6ff8-af72-46c1-a592-c631555b774f-kube-api-access-cq2bk\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.866781 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.866753 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-workload-socket\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.866858 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.866825 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-credential-socket\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.866937 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.866898 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-workload-certs\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.867030 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.867000 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-istio-data\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.867172 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.867108 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e66c6ff8-af72-46c1-a592-c631555b774f-istiod-ca-cert\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.868740 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.868720 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-istio-envoy\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.868830 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.868776 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e66c6ff8-af72-46c1-a592-c631555b774f-istio-podinfo\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.874518 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.874494 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e66c6ff8-af72-46c1-a592-c631555b774f-istio-token\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.874995 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.874965 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq2bk\" (UniqueName: \"kubernetes.io/projected/e66c6ff8-af72-46c1-a592-c631555b774f-kube-api-access-cq2bk\") pod \"e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:01.972793 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:01.972757 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:02.123406 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:02.123374 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq"] Jul 07 14:56:02.126634 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:56:02.126601 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode66c6ff8_af72_46c1_a592_c631555b774f.slice/crio-8f1c0a4b4a5fb858aa9727e360370e466428e1852d181cc12ceff693ee8c31ac WatchSource:0}: Error finding container 8f1c0a4b4a5fb858aa9727e360370e466428e1852d181cc12ceff693ee8c31ac: Status 404 returned error can't find the container with id 8f1c0a4b4a5fb858aa9727e360370e466428e1852d181cc12ceff693ee8c31ac Jul 07 14:56:02.128892 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:02.128862 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:56:02.128965 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:02.128925 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:56:02.128965 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:02.128952 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:56:02.211802 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:02.211762 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" event={"ID":"e66c6ff8-af72-46c1-a592-c631555b774f","Type":"ContainerStarted","Data":"60b8453d7089f6435a730c0f579b906fbd6646f8eaf4464d31c142ac1195d180"} Jul 07 14:56:02.211970 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:02.211833 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" event={"ID":"e66c6ff8-af72-46c1-a592-c631555b774f","Type":"ContainerStarted","Data":"8f1c0a4b4a5fb858aa9727e360370e466428e1852d181cc12ceff693ee8c31ac"} Jul 07 14:56:02.231798 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:02.231454 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" podStartSLOduration=1.231433918 podStartE2EDuration="1.231433918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:02.230748122 +0000 UTC m=+1763.578801679" watchObservedRunningTime="2026-07-07 14:56:02.231433918 +0000 UTC m=+1763.579487471" Jul 07 14:56:02.973243 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:02.973203 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:03.979473 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:03.979434 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:04.220824 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:04.220794 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:04.221802 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:04.221786 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:17.282466 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:17.282428 2555 generic.go:356] "Generic (PLEG): container finished" podID="e66c6ff8-af72-46c1-a592-c631555b774f" containerID="60b8453d7089f6435a730c0f579b906fbd6646f8eaf4464d31c142ac1195d180" exitCode=0 Jul 07 14:56:17.282904 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:17.282506 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" event={"ID":"e66c6ff8-af72-46c1-a592-c631555b774f","Type":"ContainerDied","Data":"60b8453d7089f6435a730c0f579b906fbd6646f8eaf4464d31c142ac1195d180"} Jul 07 14:56:17.283001 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:17.282984 2555 scope.go:117] "RemoveContainer" containerID="60b8453d7089f6435a730c0f579b906fbd6646f8eaf4464d31c142ac1195d180" Jul 07 14:56:17.284063 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:17.284034 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:56:17.284188 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:17.284144 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:56:17.284243 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:17.284194 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:56:18.292730 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:18.292688 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" event={"ID":"e66c6ff8-af72-46c1-a592-c631555b774f","Type":"ContainerStarted","Data":"0d345de9f412545cceac25a7fc219269893d2882df92eb93c0ed8110457a78ff"} Jul 07 14:56:18.972985 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:18.972949 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:18.973893 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:18.973865 2555 patch_prober.go:28] interesting pod/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 07 14:56:18.973996 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:18.973917 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" podUID="e66c6ff8-af72-46c1-a592-c631555b774f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 07 14:56:19.220735 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:19.220702 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:19.973720 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:19.973683 2555 patch_prober.go:28] interesting pod/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 07 14:56:19.974146 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:19.973742 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" podUID="e66c6ff8-af72-46c1-a592-c631555b774f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 07 14:56:20.973234 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:20.973200 2555 patch_prober.go:28] interesting pod/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 07 14:56:20.973423 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:20.973267 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" podUID="e66c6ff8-af72-46c1-a592-c631555b774f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 07 14:56:21.974207 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:21.974166 2555 patch_prober.go:28] interesting pod/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 07 14:56:21.974808 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:21.974775 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" podUID="e66c6ff8-af72-46c1-a592-c631555b774f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 07 14:56:22.977826 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:22.977792 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:22.978920 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:22.978902 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:24.107165 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:24.107133 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq"] Jul 07 14:56:24.324276 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:24.324237 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" podUID="e66c6ff8-af72-46c1-a592-c631555b774f" containerName="istio-proxy" containerID="cri-o://0d345de9f412545cceac25a7fc219269893d2882df92eb93c0ed8110457a78ff" gracePeriod=30 Jul 07 14:56:29.579597 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.579570 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:29.659556 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.659478 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-workload-socket\") pod \"e66c6ff8-af72-46c1-a592-c631555b774f\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " Jul 07 14:56:29.659556 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.659524 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-workload-certs\") pod \"e66c6ff8-af72-46c1-a592-c631555b774f\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " Jul 07 14:56:29.659556 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.659554 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e66c6ff8-af72-46c1-a592-c631555b774f-istiod-ca-cert\") pod \"e66c6ff8-af72-46c1-a592-c631555b774f\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " Jul 07 14:56:29.659815 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.659609 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-istio-envoy\") pod \"e66c6ff8-af72-46c1-a592-c631555b774f\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " Jul 07 14:56:29.659815 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.659637 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e66c6ff8-af72-46c1-a592-c631555b774f-istio-token\") pod \"e66c6ff8-af72-46c1-a592-c631555b774f\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " Jul 07 14:56:29.659815 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.659680 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cq2bk\" (UniqueName: \"kubernetes.io/projected/e66c6ff8-af72-46c1-a592-c631555b774f-kube-api-access-cq2bk\") pod \"e66c6ff8-af72-46c1-a592-c631555b774f\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " Jul 07 14:56:29.659815 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.659745 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e66c6ff8-af72-46c1-a592-c631555b774f-istio-podinfo\") pod \"e66c6ff8-af72-46c1-a592-c631555b774f\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " Jul 07 14:56:29.659815 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.659739 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e66c6ff8-af72-46c1-a592-c631555b774f" (UID: "e66c6ff8-af72-46c1-a592-c631555b774f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:56:29.659815 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.659781 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-istio-data\") pod \"e66c6ff8-af72-46c1-a592-c631555b774f\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " Jul 07 14:56:29.659815 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.659817 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-credential-socket\") pod \"e66c6ff8-af72-46c1-a592-c631555b774f\" (UID: \"e66c6ff8-af72-46c1-a592-c631555b774f\") " Jul 07 14:56:29.660204 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.659893 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e66c6ff8-af72-46c1-a592-c631555b774f" (UID: "e66c6ff8-af72-46c1-a592-c631555b774f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:56:29.660258 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.660219 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-workload-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:56:29.660258 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.660240 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-workload-certs\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:56:29.660258 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.660239 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e66c6ff8-af72-46c1-a592-c631555b774f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e66c6ff8-af72-46c1-a592-c631555b774f" (UID: "e66c6ff8-af72-46c1-a592-c631555b774f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:56:29.660579 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.660446 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e66c6ff8-af72-46c1-a592-c631555b774f" (UID: "e66c6ff8-af72-46c1-a592-c631555b774f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:56:29.661033 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.661007 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-istio-data" (OuterVolumeSpecName: "istio-data") pod "e66c6ff8-af72-46c1-a592-c631555b774f" (UID: "e66c6ff8-af72-46c1-a592-c631555b774f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:56:29.662258 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.662234 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e66c6ff8-af72-46c1-a592-c631555b774f-kube-api-access-cq2bk" (OuterVolumeSpecName: "kube-api-access-cq2bk") pod "e66c6ff8-af72-46c1-a592-c631555b774f" (UID: "e66c6ff8-af72-46c1-a592-c631555b774f"). InnerVolumeSpecName "kube-api-access-cq2bk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:56:29.662372 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.662339 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e66c6ff8-af72-46c1-a592-c631555b774f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e66c6ff8-af72-46c1-a592-c631555b774f" (UID: "e66c6ff8-af72-46c1-a592-c631555b774f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 14:56:29.662455 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.662428 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e66c6ff8-af72-46c1-a592-c631555b774f" (UID: "e66c6ff8-af72-46c1-a592-c631555b774f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:56:29.662455 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.662442 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e66c6ff8-af72-46c1-a592-c631555b774f-istio-token" (OuterVolumeSpecName: "istio-token") pod "e66c6ff8-af72-46c1-a592-c631555b774f" (UID: "e66c6ff8-af72-46c1-a592-c631555b774f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:56:29.761169 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.761132 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e66c6ff8-af72-46c1-a592-c631555b774f-istiod-ca-cert\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:56:29.761169 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.761161 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-istio-envoy\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:56:29.761169 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.761170 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e66c6ff8-af72-46c1-a592-c631555b774f-istio-token\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:56:29.761169 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.761178 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cq2bk\" (UniqueName: \"kubernetes.io/projected/e66c6ff8-af72-46c1-a592-c631555b774f-kube-api-access-cq2bk\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:56:29.761437 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.761187 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e66c6ff8-af72-46c1-a592-c631555b774f-istio-podinfo\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:56:29.761437 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.761196 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-istio-data\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:56:29.761437 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:29.761204 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e66c6ff8-af72-46c1-a592-c631555b774f-credential-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:56:30.354329 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:30.354295 2555 generic.go:356] "Generic (PLEG): container finished" podID="e66c6ff8-af72-46c1-a592-c631555b774f" containerID="0d345de9f412545cceac25a7fc219269893d2882df92eb93c0ed8110457a78ff" exitCode=0 Jul 07 14:56:30.354498 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:30.354344 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" Jul 07 14:56:30.354498 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:30.354383 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" event={"ID":"e66c6ff8-af72-46c1-a592-c631555b774f","Type":"ContainerDied","Data":"0d345de9f412545cceac25a7fc219269893d2882df92eb93c0ed8110457a78ff"} Jul 07 14:56:30.354498 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:30.354437 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq" event={"ID":"e66c6ff8-af72-46c1-a592-c631555b774f","Type":"ContainerDied","Data":"8f1c0a4b4a5fb858aa9727e360370e466428e1852d181cc12ceff693ee8c31ac"} Jul 07 14:56:30.354498 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:30.354459 2555 scope.go:117] "RemoveContainer" containerID="0d345de9f412545cceac25a7fc219269893d2882df92eb93c0ed8110457a78ff" Jul 07 14:56:30.364282 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:30.364266 2555 scope.go:117] "RemoveContainer" containerID="60b8453d7089f6435a730c0f579b906fbd6646f8eaf4464d31c142ac1195d180" Jul 07 14:56:30.373493 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:30.373462 2555 scope.go:117] "RemoveContainer" containerID="0d345de9f412545cceac25a7fc219269893d2882df92eb93c0ed8110457a78ff" Jul 07 14:56:30.373897 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:56:30.373876 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d345de9f412545cceac25a7fc219269893d2882df92eb93c0ed8110457a78ff\": container with ID starting with 0d345de9f412545cceac25a7fc219269893d2882df92eb93c0ed8110457a78ff not found: ID does not exist" containerID="0d345de9f412545cceac25a7fc219269893d2882df92eb93c0ed8110457a78ff" Jul 07 14:56:30.373983 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:30.373904 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d345de9f412545cceac25a7fc219269893d2882df92eb93c0ed8110457a78ff"} err="failed to get container status \"0d345de9f412545cceac25a7fc219269893d2882df92eb93c0ed8110457a78ff\": rpc error: code = NotFound desc = could not find container \"0d345de9f412545cceac25a7fc219269893d2882df92eb93c0ed8110457a78ff\": container with ID starting with 0d345de9f412545cceac25a7fc219269893d2882df92eb93c0ed8110457a78ff not found: ID does not exist" Jul 07 14:56:30.373983 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:30.373922 2555 scope.go:117] "RemoveContainer" containerID="60b8453d7089f6435a730c0f579b906fbd6646f8eaf4464d31c142ac1195d180" Jul 07 14:56:30.374259 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:56:30.374238 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60b8453d7089f6435a730c0f579b906fbd6646f8eaf4464d31c142ac1195d180\": container with ID starting with 60b8453d7089f6435a730c0f579b906fbd6646f8eaf4464d31c142ac1195d180 not found: ID does not exist" containerID="60b8453d7089f6435a730c0f579b906fbd6646f8eaf4464d31c142ac1195d180" Jul 07 14:56:30.374320 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:30.374267 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60b8453d7089f6435a730c0f579b906fbd6646f8eaf4464d31c142ac1195d180"} err="failed to get container status \"60b8453d7089f6435a730c0f579b906fbd6646f8eaf4464d31c142ac1195d180\": rpc error: code = NotFound desc = could not find container \"60b8453d7089f6435a730c0f579b906fbd6646f8eaf4464d31c142ac1195d180\": container with ID starting with 60b8453d7089f6435a730c0f579b906fbd6646f8eaf4464d31c142ac1195d180 not found: ID does not exist" Jul 07 14:56:30.375369 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:30.375349 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq"] Jul 07 14:56:30.380721 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:30.380701 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8dea50b1-openshift-default-bb96766f9-xjwgq"] Jul 07 14:56:31.179531 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:31.179500 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e66c6ff8-af72-46c1-a592-c631555b774f" path="/var/lib/kubelet/pods/e66c6ff8-af72-46c1-a592-c631555b774f/volumes" Jul 07 14:56:39.186749 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:39.186721 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:56:39.191181 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:39.191159 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 14:56:44.363928 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.363871 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl"] Jul 07 14:56:44.365359 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.365331 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e66c6ff8-af72-46c1-a592-c631555b774f" containerName="istio-proxy" Jul 07 14:56:44.365524 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.365512 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e66c6ff8-af72-46c1-a592-c631555b774f" containerName="istio-proxy" Jul 07 14:56:44.365829 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.365816 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e66c6ff8-af72-46c1-a592-c631555b774f" containerName="istio-proxy" Jul 07 14:56:44.369706 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.369680 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.372143 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.372123 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4afb43ce-openshift-default-dockercfg-kszps\"" Jul 07 14:56:44.378947 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.378920 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl"] Jul 07 14:56:44.401818 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.401783 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-workload-certs\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.401975 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.401822 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-podinfo\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.401975 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.401852 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-envoy\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.401975 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.401870 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istiod-ca-cert\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.401975 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.401904 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-token\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.401975 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.401934 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-data\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.402247 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.402049 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-credential-socket\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.402247 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.402116 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-workload-socket\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.402247 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.402144 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2sh5\" (UniqueName: \"kubernetes.io/projected/84e21e0d-032d-4ca4-b0bc-1ef36453358f-kube-api-access-v2sh5\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.503450 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.503408 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-workload-socket\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.503450 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.503450 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2sh5\" (UniqueName: \"kubernetes.io/projected/84e21e0d-032d-4ca4-b0bc-1ef36453358f-kube-api-access-v2sh5\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.503692 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.503501 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-workload-certs\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.503692 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.503524 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-podinfo\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.503692 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.503549 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-envoy\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.503894 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.503826 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istiod-ca-cert\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.503894 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.503875 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-workload-socket\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.504034 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.503940 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-token\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.504034 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.503992 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-data\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.504034 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.503901 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-workload-certs\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.504229 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.504147 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-credential-socket\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.504461 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.504442 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-data\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.504639 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.504469 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-credential-socket\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.504750 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.504588 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istiod-ca-cert\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.506219 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.506200 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-envoy\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.506351 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.506333 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-podinfo\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.512961 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.512938 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-token\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.513259 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.513239 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2sh5\" (UniqueName: \"kubernetes.io/projected/84e21e0d-032d-4ca4-b0bc-1ef36453358f-kube-api-access-v2sh5\") pod \"e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.683283 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.683203 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:44.823197 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.823175 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl"] Jul 07 14:56:44.825870 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:56:44.825840 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84e21e0d_032d_4ca4_b0bc_1ef36453358f.slice/crio-5fa3f9b8c015760fd0e05aa195ad8e39eb76e7a82a0ff34b3028290062044467 WatchSource:0}: Error finding container 5fa3f9b8c015760fd0e05aa195ad8e39eb76e7a82a0ff34b3028290062044467: Status 404 returned error can't find the container with id 5fa3f9b8c015760fd0e05aa195ad8e39eb76e7a82a0ff34b3028290062044467 Jul 07 14:56:44.828045 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.828010 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:56:44.828171 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.828101 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:56:44.828232 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:44.828172 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:56:45.424824 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:45.424788 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" event={"ID":"84e21e0d-032d-4ca4-b0bc-1ef36453358f","Type":"ContainerStarted","Data":"b5295449b6dd189ed7e65379282992986eee5356596dbd7e7865ef11ed75af33"} Jul 07 14:56:45.425202 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:45.424855 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" event={"ID":"84e21e0d-032d-4ca4-b0bc-1ef36453358f","Type":"ContainerStarted","Data":"5fa3f9b8c015760fd0e05aa195ad8e39eb76e7a82a0ff34b3028290062044467"} Jul 07 14:56:45.445999 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:45.445950 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" podStartSLOduration=1.445934391 podStartE2EDuration="1.445934391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:45.444337469 +0000 UTC m=+1806.792391022" watchObservedRunningTime="2026-07-07 14:56:45.445934391 +0000 UTC m=+1806.793987991" Jul 07 14:56:45.684027 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:45.683944 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:45.688625 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:45.688599 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:46.429556 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:46.429529 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:56:46.430482 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:56:46.430464 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:57:00.505865 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:00.505819 2555 generic.go:356] "Generic (PLEG): container finished" podID="84e21e0d-032d-4ca4-b0bc-1ef36453358f" containerID="b5295449b6dd189ed7e65379282992986eee5356596dbd7e7865ef11ed75af33" exitCode=0 Jul 07 14:57:00.506256 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:00.505919 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" event={"ID":"84e21e0d-032d-4ca4-b0bc-1ef36453358f","Type":"ContainerDied","Data":"b5295449b6dd189ed7e65379282992986eee5356596dbd7e7865ef11ed75af33"} Jul 07 14:57:00.506317 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:00.506302 2555 scope.go:117] "RemoveContainer" containerID="b5295449b6dd189ed7e65379282992986eee5356596dbd7e7865ef11ed75af33" Jul 07 14:57:00.507258 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:00.507229 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:57:00.507361 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:00.507294 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:57:00.507361 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:00.507325 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:57:01.429686 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:01.429637 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:57:01.512983 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:01.512949 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" event={"ID":"84e21e0d-032d-4ca4-b0bc-1ef36453358f","Type":"ContainerStarted","Data":"6e4b79a13f7e31616e6984e8333390eb0ec119f747bd2ee7a39d1acc8f9f58ad"} Jul 07 14:57:01.685198 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:01.685095 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:57:01.685372 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:01.685253 2555 patch_prober.go:28] interesting pod/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 07 14:57:01.685372 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:01.685306 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" podUID="84e21e0d-032d-4ca4-b0bc-1ef36453358f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 07 14:57:02.684264 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:02.684223 2555 patch_prober.go:28] interesting pod/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 07 14:57:02.684768 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:02.684302 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" podUID="84e21e0d-032d-4ca4-b0bc-1ef36453358f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 07 14:57:03.683872 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:03.683830 2555 patch_prober.go:28] interesting pod/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 07 14:57:03.684110 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:03.683903 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" podUID="84e21e0d-032d-4ca4-b0bc-1ef36453358f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 07 14:57:04.684252 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:04.684210 2555 patch_prober.go:28] interesting pod/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 07 14:57:04.684864 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:04.684830 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" podUID="84e21e0d-032d-4ca4-b0bc-1ef36453358f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 07 14:57:05.684188 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:05.684150 2555 patch_prober.go:28] interesting pod/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 07 14:57:05.684467 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:05.684439 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" podUID="84e21e0d-032d-4ca4-b0bc-1ef36453358f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 07 14:57:06.688337 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:06.688300 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:57:06.688877 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:06.688672 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:57:06.689378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:06.689358 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:57:54.920104 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:54.920053 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl"] Jul 07 14:57:54.920658 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:57:54.920356 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" podUID="84e21e0d-032d-4ca4-b0bc-1ef36453358f" containerName="istio-proxy" containerID="cri-o://6e4b79a13f7e31616e6984e8333390eb0ec119f747bd2ee7a39d1acc8f9f58ad" gracePeriod=30 Jul 07 14:58:00.171341 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.171313 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:58:00.220733 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.220709 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-token\") pod \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " Jul 07 14:58:00.220915 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.220769 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-podinfo\") pod \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " Jul 07 14:58:00.220915 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.220798 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-workload-socket\") pod \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " Jul 07 14:58:00.220915 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.220896 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-envoy\") pod \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " Jul 07 14:58:00.221110 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.220934 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-credential-socket\") pod \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " Jul 07 14:58:00.221110 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.220982 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-workload-certs\") pod \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " Jul 07 14:58:00.221110 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.221030 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istiod-ca-cert\") pod \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " Jul 07 14:58:00.221110 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.221100 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2sh5\" (UniqueName: \"kubernetes.io/projected/84e21e0d-032d-4ca4-b0bc-1ef36453358f-kube-api-access-v2sh5\") pod \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " Jul 07 14:58:00.221329 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.221109 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "84e21e0d-032d-4ca4-b0bc-1ef36453358f" (UID: "84e21e0d-032d-4ca4-b0bc-1ef36453358f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:58:00.221329 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.221144 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-data\") pod \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\" (UID: \"84e21e0d-032d-4ca4-b0bc-1ef36453358f\") " Jul 07 14:58:00.221565 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.221468 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-workload-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:58:00.221565 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.221521 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "84e21e0d-032d-4ca4-b0bc-1ef36453358f" (UID: "84e21e0d-032d-4ca4-b0bc-1ef36453358f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:58:00.221565 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.221518 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "84e21e0d-032d-4ca4-b0bc-1ef36453358f" (UID: "84e21e0d-032d-4ca4-b0bc-1ef36453358f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 14:58:00.221800 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.221548 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "84e21e0d-032d-4ca4-b0bc-1ef36453358f" (UID: "84e21e0d-032d-4ca4-b0bc-1ef36453358f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:58:00.222450 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.222426 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-data" (OuterVolumeSpecName: "istio-data") pod "84e21e0d-032d-4ca4-b0bc-1ef36453358f" (UID: "84e21e0d-032d-4ca4-b0bc-1ef36453358f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:58:00.223518 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.223485 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84e21e0d-032d-4ca4-b0bc-1ef36453358f-kube-api-access-v2sh5" (OuterVolumeSpecName: "kube-api-access-v2sh5") pod "84e21e0d-032d-4ca4-b0bc-1ef36453358f" (UID: "84e21e0d-032d-4ca4-b0bc-1ef36453358f"). InnerVolumeSpecName "kube-api-access-v2sh5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:58:00.223621 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.223579 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "84e21e0d-032d-4ca4-b0bc-1ef36453358f" (UID: "84e21e0d-032d-4ca4-b0bc-1ef36453358f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 14:58:00.223621 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.223591 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "84e21e0d-032d-4ca4-b0bc-1ef36453358f" (UID: "84e21e0d-032d-4ca4-b0bc-1ef36453358f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 14:58:00.223708 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.223621 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-token" (OuterVolumeSpecName: "istio-token") pod "84e21e0d-032d-4ca4-b0bc-1ef36453358f" (UID: "84e21e0d-032d-4ca4-b0bc-1ef36453358f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 14:58:00.322528 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.322495 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-data\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:58:00.322528 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.322521 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-token\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:58:00.322528 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.322530 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-podinfo\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:58:00.322528 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.322538 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istio-envoy\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:58:00.322785 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.322546 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-credential-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:58:00.322785 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.322556 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84e21e0d-032d-4ca4-b0bc-1ef36453358f-workload-certs\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:58:00.322785 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.322564 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84e21e0d-032d-4ca4-b0bc-1ef36453358f-istiod-ca-cert\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:58:00.322785 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.322573 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2sh5\" (UniqueName: \"kubernetes.io/projected/84e21e0d-032d-4ca4-b0bc-1ef36453358f-kube-api-access-v2sh5\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 14:58:00.794278 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.794241 2555 generic.go:356] "Generic (PLEG): container finished" podID="84e21e0d-032d-4ca4-b0bc-1ef36453358f" containerID="6e4b79a13f7e31616e6984e8333390eb0ec119f747bd2ee7a39d1acc8f9f58ad" exitCode=0 Jul 07 14:58:00.794454 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.794294 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" Jul 07 14:58:00.794454 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.794325 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" event={"ID":"84e21e0d-032d-4ca4-b0bc-1ef36453358f","Type":"ContainerDied","Data":"6e4b79a13f7e31616e6984e8333390eb0ec119f747bd2ee7a39d1acc8f9f58ad"} Jul 07 14:58:00.794454 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.794378 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl" event={"ID":"84e21e0d-032d-4ca4-b0bc-1ef36453358f","Type":"ContainerDied","Data":"5fa3f9b8c015760fd0e05aa195ad8e39eb76e7a82a0ff34b3028290062044467"} Jul 07 14:58:00.794454 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.794399 2555 scope.go:117] "RemoveContainer" containerID="6e4b79a13f7e31616e6984e8333390eb0ec119f747bd2ee7a39d1acc8f9f58ad" Jul 07 14:58:00.803776 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.803756 2555 scope.go:117] "RemoveContainer" containerID="b5295449b6dd189ed7e65379282992986eee5356596dbd7e7865ef11ed75af33" Jul 07 14:58:00.811956 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.811942 2555 scope.go:117] "RemoveContainer" containerID="6e4b79a13f7e31616e6984e8333390eb0ec119f747bd2ee7a39d1acc8f9f58ad" Jul 07 14:58:00.812267 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:58:00.812249 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e4b79a13f7e31616e6984e8333390eb0ec119f747bd2ee7a39d1acc8f9f58ad\": container with ID starting with 6e4b79a13f7e31616e6984e8333390eb0ec119f747bd2ee7a39d1acc8f9f58ad not found: ID does not exist" containerID="6e4b79a13f7e31616e6984e8333390eb0ec119f747bd2ee7a39d1acc8f9f58ad" Jul 07 14:58:00.812322 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.812274 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e4b79a13f7e31616e6984e8333390eb0ec119f747bd2ee7a39d1acc8f9f58ad"} err="failed to get container status \"6e4b79a13f7e31616e6984e8333390eb0ec119f747bd2ee7a39d1acc8f9f58ad\": rpc error: code = NotFound desc = could not find container \"6e4b79a13f7e31616e6984e8333390eb0ec119f747bd2ee7a39d1acc8f9f58ad\": container with ID starting with 6e4b79a13f7e31616e6984e8333390eb0ec119f747bd2ee7a39d1acc8f9f58ad not found: ID does not exist" Jul 07 14:58:00.812322 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.812291 2555 scope.go:117] "RemoveContainer" containerID="b5295449b6dd189ed7e65379282992986eee5356596dbd7e7865ef11ed75af33" Jul 07 14:58:00.812517 ip-10-0-141-129 kubenswrapper[2555]: E0707 14:58:00.812499 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5295449b6dd189ed7e65379282992986eee5356596dbd7e7865ef11ed75af33\": container with ID starting with b5295449b6dd189ed7e65379282992986eee5356596dbd7e7865ef11ed75af33 not found: ID does not exist" containerID="b5295449b6dd189ed7e65379282992986eee5356596dbd7e7865ef11ed75af33" Jul 07 14:58:00.812557 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.812524 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5295449b6dd189ed7e65379282992986eee5356596dbd7e7865ef11ed75af33"} err="failed to get container status \"b5295449b6dd189ed7e65379282992986eee5356596dbd7e7865ef11ed75af33\": rpc error: code = NotFound desc = could not find container \"b5295449b6dd189ed7e65379282992986eee5356596dbd7e7865ef11ed75af33\": container with ID starting with b5295449b6dd189ed7e65379282992986eee5356596dbd7e7865ef11ed75af33 not found: ID does not exist" Jul 07 14:58:00.823267 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.823247 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl"] Jul 07 14:58:00.824828 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:00.824807 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4afb43ce-openshift-default-5d58d67899-42jjl"] Jul 07 14:58:01.181135 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:01.181014 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84e21e0d-032d-4ca4-b0bc-1ef36453358f" path="/var/lib/kubelet/pods/84e21e0d-032d-4ca4-b0bc-1ef36453358f/volumes" Jul 07 14:58:52.225479 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.225441 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks"] Jul 07 14:58:52.225875 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.225847 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e66c6ff8-af72-46c1-a592-c631555b774f" containerName="istio-proxy" Jul 07 14:58:52.225875 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.225864 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e66c6ff8-af72-46c1-a592-c631555b774f" containerName="istio-proxy" Jul 07 14:58:52.225951 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.225880 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84e21e0d-032d-4ca4-b0bc-1ef36453358f" containerName="istio-proxy" Jul 07 14:58:52.225951 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.225886 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e21e0d-032d-4ca4-b0bc-1ef36453358f" containerName="istio-proxy" Jul 07 14:58:52.225951 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.225894 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84e21e0d-032d-4ca4-b0bc-1ef36453358f" containerName="istio-proxy" Jul 07 14:58:52.225951 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.225899 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e21e0d-032d-4ca4-b0bc-1ef36453358f" containerName="istio-proxy" Jul 07 14:58:52.226099 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.225960 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="84e21e0d-032d-4ca4-b0bc-1ef36453358f" containerName="istio-proxy" Jul 07 14:58:52.226099 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.225969 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e66c6ff8-af72-46c1-a592-c631555b774f" containerName="istio-proxy" Jul 07 14:58:52.228999 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.228981 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.231128 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.231104 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-17aa9c-openshift-default-dockercfg-8wjgr\"" Jul 07 14:58:52.242848 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.242805 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks"] Jul 07 14:58:52.308559 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.308524 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-workload-socket\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.308741 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.308574 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-envoy\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.308741 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.308641 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-token\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.308844 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.308767 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-credential-socket\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.308844 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.308796 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-data\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.308844 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.308815 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istiod-ca-cert\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.308844 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.308831 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r9p7\" (UniqueName: \"kubernetes.io/projected/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-kube-api-access-4r9p7\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.309001 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.308869 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-podinfo\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.309001 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.308900 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-workload-certs\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.410106 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.410061 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-credential-socket\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.410106 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.410110 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-data\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.410362 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.410129 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istiod-ca-cert\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.410362 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.410149 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r9p7\" (UniqueName: \"kubernetes.io/projected/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-kube-api-access-4r9p7\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.410362 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.410174 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-podinfo\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.410362 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.410197 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-workload-certs\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.410362 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.410220 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-workload-socket\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.410362 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.410246 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-envoy\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.410362 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.410342 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-token\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.410675 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.410544 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-data\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.410788 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.410752 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-workload-certs\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.410916 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.410809 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-workload-socket\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.410916 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.410876 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-credential-socket\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.411135 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.411119 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istiod-ca-cert\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.412868 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.412843 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-podinfo\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.413160 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.413144 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-envoy\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.423030 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.423003 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r9p7\" (UniqueName: \"kubernetes.io/projected/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-kube-api-access-4r9p7\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.423231 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.423208 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-token\") pod \"e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.538904 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.538822 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:52.688658 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.688479 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks"] Jul 07 14:58:52.694101 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:58:52.691933 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b9102b0_e608_40d8_a65b_a6cb8996ee9b.slice/crio-e9c61cc80116a471ce24a2f9986f923241b005a93beee84464f7fa9281144185 WatchSource:0}: Error finding container e9c61cc80116a471ce24a2f9986f923241b005a93beee84464f7fa9281144185: Status 404 returned error can't find the container with id e9c61cc80116a471ce24a2f9986f923241b005a93beee84464f7fa9281144185 Jul 07 14:58:52.695879 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.695846 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:58:52.695972 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.695916 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:58:52.695972 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:52.695946 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:58:53.036540 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:53.036501 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" event={"ID":"3b9102b0-e608-40d8-a65b-a6cb8996ee9b","Type":"ContainerStarted","Data":"b4900e05ea68f12c981ebb3ef721c44ae83b0b2c518ad7fcc5e32473122f61f0"} Jul 07 14:58:53.036704 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:53.036562 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" event={"ID":"3b9102b0-e608-40d8-a65b-a6cb8996ee9b","Type":"ContainerStarted","Data":"e9c61cc80116a471ce24a2f9986f923241b005a93beee84464f7fa9281144185"} Jul 07 14:58:53.063548 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:53.063501 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" podStartSLOduration=1.063485645 podStartE2EDuration="1.063485645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:58:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:58:53.06284656 +0000 UTC m=+1934.410900113" watchObservedRunningTime="2026-07-07 14:58:53.063485645 +0000 UTC m=+1934.411539199" Jul 07 14:58:53.539707 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:53.539666 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:53.544591 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:53.544566 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:54.041360 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:54.041324 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:58:54.042246 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:58:54.042229 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:59:00.318315 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.318280 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4"] Jul 07 14:59:00.318754 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.318740 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="84e21e0d-032d-4ca4-b0bc-1ef36453358f" containerName="istio-proxy" Jul 07 14:59:00.321919 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.321904 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.324519 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.324497 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-54f2c7-openshift-default-dockercfg-4c8fm\"" Jul 07 14:59:00.351655 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.351625 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4"] Jul 07 14:59:00.386109 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.386080 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-data\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.386238 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.386155 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-workload-socket\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.386238 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.386222 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm6cw\" (UniqueName: \"kubernetes.io/projected/ae40b01e-3ce0-427e-a7b2-9e8739f17165-kube-api-access-qm6cw\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.386340 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.386245 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-envoy\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.386340 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.386263 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-podinfo\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.386340 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.386279 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istiod-ca-cert\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.386448 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.386390 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-credential-socket\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.386448 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.386430 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-workload-certs\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.386522 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.386476 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-token\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.487514 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.487481 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-envoy\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.487705 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.487559 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-podinfo\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.487705 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.487592 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istiod-ca-cert\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.487705 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.487649 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-credential-socket\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.487895 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.487802 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-workload-certs\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.487895 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.487885 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-token\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.488003 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.487947 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-data\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.488059 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.488012 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-workload-socket\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.488139 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.488115 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm6cw\" (UniqueName: \"kubernetes.io/projected/ae40b01e-3ce0-427e-a7b2-9e8739f17165-kube-api-access-qm6cw\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.488643 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.488218 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-credential-socket\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.488643 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.488248 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-workload-certs\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.488643 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.488486 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-workload-socket\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.488643 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.488576 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-data\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.489044 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.489018 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istiod-ca-cert\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.490451 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.490429 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-envoy\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.490672 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.490654 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-podinfo\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.498958 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.498938 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-token\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.500153 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.500131 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm6cw\" (UniqueName: \"kubernetes.io/projected/ae40b01e-3ce0-427e-a7b2-9e8739f17165-kube-api-access-qm6cw\") pod \"e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.633365 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.633285 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:00.778177 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.778150 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4"] Jul 07 14:59:00.781297 ip-10-0-141-129 kubenswrapper[2555]: W0707 14:59:00.781269 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae40b01e_3ce0_427e_a7b2_9e8739f17165.slice/crio-e2aa91f550948c7d9d9f15a2a6daeafb34f5232d50e055fcafbcbc1bcda1068a WatchSource:0}: Error finding container e2aa91f550948c7d9d9f15a2a6daeafb34f5232d50e055fcafbcbc1bcda1068a: Status 404 returned error can't find the container with id e2aa91f550948c7d9d9f15a2a6daeafb34f5232d50e055fcafbcbc1bcda1068a Jul 07 14:59:00.783417 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.783384 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:59:00.783490 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.783448 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:59:00.783490 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:00.783475 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:59:01.074842 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:01.074803 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" event={"ID":"ae40b01e-3ce0-427e-a7b2-9e8739f17165","Type":"ContainerStarted","Data":"f6344a84ee33847d6096c46d8469a1ebca909c93e033ed8b15768c252717ebdf"} Jul 07 14:59:01.075143 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:01.075113 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" event={"ID":"ae40b01e-3ce0-427e-a7b2-9e8739f17165","Type":"ContainerStarted","Data":"e2aa91f550948c7d9d9f15a2a6daeafb34f5232d50e055fcafbcbc1bcda1068a"} Jul 07 14:59:01.096903 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:01.096844 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" podStartSLOduration=1.096822854 podStartE2EDuration="1.096822854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:59:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:59:01.095433174 +0000 UTC m=+1942.443486738" watchObservedRunningTime="2026-07-07 14:59:01.096822854 +0000 UTC m=+1942.444876409" Jul 07 14:59:01.633850 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:01.633811 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:01.638582 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:01.638560 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:02.080228 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:02.080188 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:02.081367 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:02.081342 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:10.083298 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:10.083243 2555 patch_prober.go:28] interesting pod/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 14:59:10.083901 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:10.083320 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" podUID="3b9102b0-e608-40d8-a65b-a6cb8996ee9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 14:59:12.146082 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:12.146035 2555 generic.go:356] "Generic (PLEG): container finished" podID="3b9102b0-e608-40d8-a65b-a6cb8996ee9b" containerID="b4900e05ea68f12c981ebb3ef721c44ae83b0b2c518ad7fcc5e32473122f61f0" exitCode=0 Jul 07 14:59:12.146601 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:12.146112 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" event={"ID":"3b9102b0-e608-40d8-a65b-a6cb8996ee9b","Type":"ContainerDied","Data":"b4900e05ea68f12c981ebb3ef721c44ae83b0b2c518ad7fcc5e32473122f61f0"} Jul 07 14:59:12.146672 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:12.146630 2555 scope.go:117] "RemoveContainer" containerID="b4900e05ea68f12c981ebb3ef721c44ae83b0b2c518ad7fcc5e32473122f61f0" Jul 07 14:59:12.147681 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:12.147655 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:59:12.147747 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:12.147724 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:59:12.147791 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:12.147756 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:59:13.152787 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:13.152750 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" event={"ID":"3b9102b0-e608-40d8-a65b-a6cb8996ee9b","Type":"ContainerStarted","Data":"7ee8b047c1f980eb4451a89bd487bd8009f1040c87ca8f94d400cd6d3e32809c"} Jul 07 14:59:13.539969 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:13.539928 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:59:13.540334 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:13.540297 2555 patch_prober.go:28] interesting pod/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 07 14:59:13.540400 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:13.540378 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" podUID="3b9102b0-e608-40d8-a65b-a6cb8996ee9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 07 14:59:14.539583 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:14.539545 2555 patch_prober.go:28] interesting pod/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 07 14:59:14.540100 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:14.539611 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" podUID="3b9102b0-e608-40d8-a65b-a6cb8996ee9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 07 14:59:15.539495 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:15.539454 2555 patch_prober.go:28] interesting pod/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 07 14:59:15.539687 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:15.539529 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" podUID="3b9102b0-e608-40d8-a65b-a6cb8996ee9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 07 14:59:16.539797 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:16.539748 2555 patch_prober.go:28] interesting pod/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 07 14:59:16.540330 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:16.539818 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" podUID="3b9102b0-e608-40d8-a65b-a6cb8996ee9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 07 14:59:17.542378 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:17.542347 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:59:17.542743 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:17.542621 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:59:17.543432 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:17.543409 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 14:59:18.122246 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:18.122207 2555 patch_prober.go:28] interesting pod/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 14:59:18.122519 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:18.122279 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" podUID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 14:59:19.191478 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:19.191385 2555 generic.go:356] "Generic (PLEG): container finished" podID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" containerID="f6344a84ee33847d6096c46d8469a1ebca909c93e033ed8b15768c252717ebdf" exitCode=0 Jul 07 14:59:19.191478 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:19.191461 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" event={"ID":"ae40b01e-3ce0-427e-a7b2-9e8739f17165","Type":"ContainerDied","Data":"f6344a84ee33847d6096c46d8469a1ebca909c93e033ed8b15768c252717ebdf"} Jul 07 14:59:19.191918 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:19.191899 2555 scope.go:117] "RemoveContainer" containerID="f6344a84ee33847d6096c46d8469a1ebca909c93e033ed8b15768c252717ebdf" Jul 07 14:59:19.192979 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:19.192952 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:59:19.193045 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:19.193018 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:59:19.193104 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:19.193051 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 14:59:20.198444 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:20.198409 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" event={"ID":"ae40b01e-3ce0-427e-a7b2-9e8739f17165","Type":"ContainerStarted","Data":"7b3d1b78e646e3fa2fcd73a0903957461ed9da124c8449b4e0a894895cefffa9"} Jul 07 14:59:20.635019 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:20.634483 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:20.635019 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:20.634660 2555 patch_prober.go:28] interesting pod/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 07 14:59:20.635019 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:20.634735 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" podUID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 07 14:59:21.634460 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:21.634420 2555 patch_prober.go:28] interesting pod/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 07 14:59:21.634862 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:21.634481 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" podUID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 07 14:59:22.634629 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:22.634588 2555 patch_prober.go:28] interesting pod/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 07 14:59:22.635124 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:22.634656 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" podUID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 07 14:59:23.633860 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:23.633815 2555 patch_prober.go:28] interesting pod/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 07 14:59:23.634036 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:23.633895 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" podUID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 07 14:59:24.634512 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:24.634470 2555 patch_prober.go:28] interesting pod/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 07 14:59:24.634888 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:24.634543 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" podUID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 07 14:59:25.637429 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:25.637401 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:25.637906 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:25.637780 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 14:59:25.638557 ip-10-0-141-129 kubenswrapper[2555]: I0707 14:59:25.638540 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 15:00:50.679523 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:50.679488 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks"] Jul 07 15:00:50.680049 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:50.679769 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" podUID="3b9102b0-e608-40d8-a65b-a6cb8996ee9b" containerName="istio-proxy" containerID="cri-o://7ee8b047c1f980eb4451a89bd487bd8009f1040c87ca8f94d400cd6d3e32809c" gracePeriod=30 Jul 07 15:00:55.936402 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:55.936377 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 15:00:56.017793 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.017758 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-envoy\") pod \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " Jul 07 15:00:56.017952 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.017803 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-credential-socket\") pod \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " Jul 07 15:00:56.017952 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.017843 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r9p7\" (UniqueName: \"kubernetes.io/projected/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-kube-api-access-4r9p7\") pod \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " Jul 07 15:00:56.017952 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.017929 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-workload-socket\") pod \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " Jul 07 15:00:56.018113 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.017978 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-token\") pod \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " Jul 07 15:00:56.018113 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.018006 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-data\") pod \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " Jul 07 15:00:56.018113 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.018033 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-podinfo\") pod \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " Jul 07 15:00:56.018113 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.018111 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istiod-ca-cert\") pod \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " Jul 07 15:00:56.018340 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.018145 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-workload-certs\") pod \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\" (UID: \"3b9102b0-e608-40d8-a65b-a6cb8996ee9b\") " Jul 07 15:00:56.018340 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.018207 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b9102b0-e608-40d8-a65b-a6cb8996ee9b" (UID: "3b9102b0-e608-40d8-a65b-a6cb8996ee9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:00:56.018340 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.018282 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b9102b0-e608-40d8-a65b-a6cb8996ee9b" (UID: "3b9102b0-e608-40d8-a65b-a6cb8996ee9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:00:56.018557 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.018509 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b9102b0-e608-40d8-a65b-a6cb8996ee9b" (UID: "3b9102b0-e608-40d8-a65b-a6cb8996ee9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:00:56.018621 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.018505 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b9102b0-e608-40d8-a65b-a6cb8996ee9b" (UID: "3b9102b0-e608-40d8-a65b-a6cb8996ee9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:00:56.018678 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.018554 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-workload-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:00:56.018678 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.018667 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-credential-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:00:56.018895 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.018872 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b9102b0-e608-40d8-a65b-a6cb8996ee9b" (UID: "3b9102b0-e608-40d8-a65b-a6cb8996ee9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:00:56.020185 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.020152 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b9102b0-e608-40d8-a65b-a6cb8996ee9b" (UID: "3b9102b0-e608-40d8-a65b-a6cb8996ee9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:00:56.020425 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.020403 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-kube-api-access-4r9p7" (OuterVolumeSpecName: "kube-api-access-4r9p7") pod "3b9102b0-e608-40d8-a65b-a6cb8996ee9b" (UID: "3b9102b0-e608-40d8-a65b-a6cb8996ee9b"). InnerVolumeSpecName "kube-api-access-4r9p7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:00:56.020509 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.020491 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b9102b0-e608-40d8-a65b-a6cb8996ee9b" (UID: "3b9102b0-e608-40d8-a65b-a6cb8996ee9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:00:56.020629 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.020611 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b9102b0-e608-40d8-a65b-a6cb8996ee9b" (UID: "3b9102b0-e608-40d8-a65b-a6cb8996ee9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:00:56.119148 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.119110 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-token\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:00:56.119148 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.119138 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-data\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:00:56.119148 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.119147 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-podinfo\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:00:56.119148 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.119155 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istiod-ca-cert\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:00:56.119408 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.119164 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-workload-certs\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:00:56.119408 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.119172 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-istio-envoy\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:00:56.119408 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.119179 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r9p7\" (UniqueName: \"kubernetes.io/projected/3b9102b0-e608-40d8-a65b-a6cb8996ee9b-kube-api-access-4r9p7\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:00:56.660963 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.660927 2555 generic.go:356] "Generic (PLEG): container finished" podID="3b9102b0-e608-40d8-a65b-a6cb8996ee9b" containerID="7ee8b047c1f980eb4451a89bd487bd8009f1040c87ca8f94d400cd6d3e32809c" exitCode=0 Jul 07 15:00:56.661178 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.660980 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" Jul 07 15:00:56.661178 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.661007 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" event={"ID":"3b9102b0-e608-40d8-a65b-a6cb8996ee9b","Type":"ContainerDied","Data":"7ee8b047c1f980eb4451a89bd487bd8009f1040c87ca8f94d400cd6d3e32809c"} Jul 07 15:00:56.661178 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.661062 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks" event={"ID":"3b9102b0-e608-40d8-a65b-a6cb8996ee9b","Type":"ContainerDied","Data":"e9c61cc80116a471ce24a2f9986f923241b005a93beee84464f7fa9281144185"} Jul 07 15:00:56.661178 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.661113 2555 scope.go:117] "RemoveContainer" containerID="7ee8b047c1f980eb4451a89bd487bd8009f1040c87ca8f94d400cd6d3e32809c" Jul 07 15:00:56.670519 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.670503 2555 scope.go:117] "RemoveContainer" containerID="b4900e05ea68f12c981ebb3ef721c44ae83b0b2c518ad7fcc5e32473122f61f0" Jul 07 15:00:56.678917 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.678895 2555 scope.go:117] "RemoveContainer" containerID="7ee8b047c1f980eb4451a89bd487bd8009f1040c87ca8f94d400cd6d3e32809c" Jul 07 15:00:56.679194 ip-10-0-141-129 kubenswrapper[2555]: E0707 15:00:56.679176 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ee8b047c1f980eb4451a89bd487bd8009f1040c87ca8f94d400cd6d3e32809c\": container with ID starting with 7ee8b047c1f980eb4451a89bd487bd8009f1040c87ca8f94d400cd6d3e32809c not found: ID does not exist" containerID="7ee8b047c1f980eb4451a89bd487bd8009f1040c87ca8f94d400cd6d3e32809c" Jul 07 15:00:56.679265 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.679202 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ee8b047c1f980eb4451a89bd487bd8009f1040c87ca8f94d400cd6d3e32809c"} err="failed to get container status \"7ee8b047c1f980eb4451a89bd487bd8009f1040c87ca8f94d400cd6d3e32809c\": rpc error: code = NotFound desc = could not find container \"7ee8b047c1f980eb4451a89bd487bd8009f1040c87ca8f94d400cd6d3e32809c\": container with ID starting with 7ee8b047c1f980eb4451a89bd487bd8009f1040c87ca8f94d400cd6d3e32809c not found: ID does not exist" Jul 07 15:00:56.679265 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.679219 2555 scope.go:117] "RemoveContainer" containerID="b4900e05ea68f12c981ebb3ef721c44ae83b0b2c518ad7fcc5e32473122f61f0" Jul 07 15:00:56.679466 ip-10-0-141-129 kubenswrapper[2555]: E0707 15:00:56.679447 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4900e05ea68f12c981ebb3ef721c44ae83b0b2c518ad7fcc5e32473122f61f0\": container with ID starting with b4900e05ea68f12c981ebb3ef721c44ae83b0b2c518ad7fcc5e32473122f61f0 not found: ID does not exist" containerID="b4900e05ea68f12c981ebb3ef721c44ae83b0b2c518ad7fcc5e32473122f61f0" Jul 07 15:00:56.679522 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.679475 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4900e05ea68f12c981ebb3ef721c44ae83b0b2c518ad7fcc5e32473122f61f0"} err="failed to get container status \"b4900e05ea68f12c981ebb3ef721c44ae83b0b2c518ad7fcc5e32473122f61f0\": rpc error: code = NotFound desc = could not find container \"b4900e05ea68f12c981ebb3ef721c44ae83b0b2c518ad7fcc5e32473122f61f0\": container with ID starting with b4900e05ea68f12c981ebb3ef721c44ae83b0b2c518ad7fcc5e32473122f61f0 not found: ID does not exist" Jul 07 15:00:56.682446 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.682425 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks"] Jul 07 15:00:56.686214 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:56.686193 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-17aa9c-openshift-default-55ccfd4cd9-7k2ks"] Jul 07 15:00:57.181645 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:57.181610 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b9102b0-e608-40d8-a65b-a6cb8996ee9b" path="/var/lib/kubelet/pods/3b9102b0-e608-40d8-a65b-a6cb8996ee9b/volumes" Jul 07 15:00:59.023061 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:59.023030 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4"] Jul 07 15:00:59.023503 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:59.023230 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" podUID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" containerName="istio-proxy" containerID="cri-o://7b3d1b78e646e3fa2fcd73a0903957461ed9da124c8449b4e0a894895cefffa9" gracePeriod=30 Jul 07 15:00:59.943979 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:59.943784 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv"] Jul 07 15:00:59.944354 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:59.944331 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b9102b0-e608-40d8-a65b-a6cb8996ee9b" containerName="istio-proxy" Jul 07 15:00:59.944449 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:59.944357 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b9102b0-e608-40d8-a65b-a6cb8996ee9b" containerName="istio-proxy" Jul 07 15:00:59.944449 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:59.944377 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b9102b0-e608-40d8-a65b-a6cb8996ee9b" containerName="istio-proxy" Jul 07 15:00:59.944449 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:59.944387 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b9102b0-e608-40d8-a65b-a6cb8996ee9b" containerName="istio-proxy" Jul 07 15:00:59.944612 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:59.944477 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b9102b0-e608-40d8-a65b-a6cb8996ee9b" containerName="istio-proxy" Jul 07 15:00:59.944612 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:59.944491 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b9102b0-e608-40d8-a65b-a6cb8996ee9b" containerName="istio-proxy" Jul 07 15:00:59.947764 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:59.947742 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:00:59.950836 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:59.950815 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-18bb73-openshift-default-dockercfg-jrkxj\"" Jul 07 15:00:59.968669 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:00:59.968646 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv"] Jul 07 15:01:00.058315 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.058268 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hhm6\" (UniqueName: \"kubernetes.io/projected/0c9ab688-6be2-4bce-8509-d874e98e272b-kube-api-access-6hhm6\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.058725 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.058423 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c9ab688-6be2-4bce-8509-d874e98e272b-istiod-ca-cert\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.058725 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.058477 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-credential-socket\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.058725 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.058510 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-workload-certs\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.058725 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.058599 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-data\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.058725 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.058636 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-workload-socket\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.058725 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.058677 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-token\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.058725 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.058722 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-envoy\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.058997 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.058788 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-podinfo\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.160261 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.160225 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-data\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.160261 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.160266 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-workload-socket\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.160513 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.160291 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-token\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.160513 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.160311 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-envoy\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.160513 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.160340 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-podinfo\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.160513 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.160372 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hhm6\" (UniqueName: \"kubernetes.io/projected/0c9ab688-6be2-4bce-8509-d874e98e272b-kube-api-access-6hhm6\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.160724 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.160522 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c9ab688-6be2-4bce-8509-d874e98e272b-istiod-ca-cert\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.160724 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.160588 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-credential-socket\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.160724 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.160623 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-workload-certs\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.160724 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.160703 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-workload-socket\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.160977 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.160954 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-credential-socket\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.161042 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.160959 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-workload-certs\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.161253 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.161229 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-data\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.161356 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.161284 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c9ab688-6be2-4bce-8509-d874e98e272b-istiod-ca-cert\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.162886 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.162865 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-podinfo\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.162967 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.162899 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-envoy\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.171734 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.171707 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-token\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.171815 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.171802 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hhm6\" (UniqueName: \"kubernetes.io/projected/0c9ab688-6be2-4bce-8509-d874e98e272b-kube-api-access-6hhm6\") pod \"e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.258400 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.258373 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:00.402659 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.402625 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv"] Jul 07 15:01:00.403996 ip-10-0-141-129 kubenswrapper[2555]: W0707 15:01:00.403971 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c9ab688_6be2_4bce_8509_d874e98e272b.slice/crio-7cafa7117bdf5ecf1fac149f8d63d16496b03e00c3edffc2ac75cb55f66b2234 WatchSource:0}: Error finding container 7cafa7117bdf5ecf1fac149f8d63d16496b03e00c3edffc2ac75cb55f66b2234: Status 404 returned error can't find the container with id 7cafa7117bdf5ecf1fac149f8d63d16496b03e00c3edffc2ac75cb55f66b2234 Jul 07 15:01:00.405940 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.405912 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:00.406006 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.405972 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:00.406006 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.406001 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:00.682843 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.682754 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" event={"ID":"0c9ab688-6be2-4bce-8509-d874e98e272b","Type":"ContainerStarted","Data":"06d323ec9b80448d0a854bbc0d669fc26547d8eeb60c3de2b67f894fcdf25077"} Jul 07 15:01:00.682998 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.682848 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" event={"ID":"0c9ab688-6be2-4bce-8509-d874e98e272b","Type":"ContainerStarted","Data":"7cafa7117bdf5ecf1fac149f8d63d16496b03e00c3edffc2ac75cb55f66b2234"} Jul 07 15:01:00.707399 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:00.707346 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" podStartSLOduration=1.707330535 podStartE2EDuration="1.707330535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:00:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:00.705305827 +0000 UTC m=+2062.053359419" watchObservedRunningTime="2026-07-07 15:01:00.707330535 +0000 UTC m=+2062.055384148" Jul 07 15:01:01.258680 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:01.258649 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:01.263385 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:01.263360 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:01.688261 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:01.688177 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:01.689020 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:01.689002 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:04.271665 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.271641 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 15:01:04.405852 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.405756 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-podinfo\") pod \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " Jul 07 15:01:04.405852 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.405797 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-workload-certs\") pod \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " Jul 07 15:01:04.405852 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.405834 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm6cw\" (UniqueName: \"kubernetes.io/projected/ae40b01e-3ce0-427e-a7b2-9e8739f17165-kube-api-access-qm6cw\") pod \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " Jul 07 15:01:04.405852 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.405853 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-envoy\") pod \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " Jul 07 15:01:04.406203 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.405870 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-token\") pod \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " Jul 07 15:01:04.406203 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.405891 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-workload-socket\") pod \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " Jul 07 15:01:04.406203 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.405920 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-credential-socket\") pod \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " Jul 07 15:01:04.406203 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.405974 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-data\") pod \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " Jul 07 15:01:04.406203 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.406015 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istiod-ca-cert\") pod \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\" (UID: \"ae40b01e-3ce0-427e-a7b2-9e8739f17165\") " Jul 07 15:01:04.406460 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.406290 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ae40b01e-3ce0-427e-a7b2-9e8739f17165" (UID: "ae40b01e-3ce0-427e-a7b2-9e8739f17165"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:01:04.406460 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.406376 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ae40b01e-3ce0-427e-a7b2-9e8739f17165" (UID: "ae40b01e-3ce0-427e-a7b2-9e8739f17165"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:01:04.406460 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.406412 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-credential-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:04.406617 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.406594 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ae40b01e-3ce0-427e-a7b2-9e8739f17165" (UID: "ae40b01e-3ce0-427e-a7b2-9e8739f17165"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:01:04.406695 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.406672 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ae40b01e-3ce0-427e-a7b2-9e8739f17165" (UID: "ae40b01e-3ce0-427e-a7b2-9e8739f17165"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:01:04.407702 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.407669 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-data" (OuterVolumeSpecName: "istio-data") pod "ae40b01e-3ce0-427e-a7b2-9e8739f17165" (UID: "ae40b01e-3ce0-427e-a7b2-9e8739f17165"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:01:04.408549 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.408521 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae40b01e-3ce0-427e-a7b2-9e8739f17165-kube-api-access-qm6cw" (OuterVolumeSpecName: "kube-api-access-qm6cw") pod "ae40b01e-3ce0-427e-a7b2-9e8739f17165" (UID: "ae40b01e-3ce0-427e-a7b2-9e8739f17165"). InnerVolumeSpecName "kube-api-access-qm6cw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:01:04.408665 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.408576 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae40b01e-3ce0-427e-a7b2-9e8739f17165" (UID: "ae40b01e-3ce0-427e-a7b2-9e8739f17165"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:01:04.408734 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.408680 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ae40b01e-3ce0-427e-a7b2-9e8739f17165" (UID: "ae40b01e-3ce0-427e-a7b2-9e8739f17165"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:01:04.408734 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.408687 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ae40b01e-3ce0-427e-a7b2-9e8739f17165" (UID: "ae40b01e-3ce0-427e-a7b2-9e8739f17165"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:01:04.507492 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.507461 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-podinfo\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:04.507492 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.507487 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-workload-certs\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:04.507492 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.507497 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm6cw\" (UniqueName: \"kubernetes.io/projected/ae40b01e-3ce0-427e-a7b2-9e8739f17165-kube-api-access-qm6cw\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:04.507721 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.507506 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-envoy\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:04.507721 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.507513 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-token\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:04.507721 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.507520 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-workload-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:04.507721 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.507528 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istio-data\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:04.507721 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.507535 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae40b01e-3ce0-427e-a7b2-9e8739f17165-istiod-ca-cert\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:04.704168 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.704053 2555 generic.go:356] "Generic (PLEG): container finished" podID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" containerID="7b3d1b78e646e3fa2fcd73a0903957461ed9da124c8449b4e0a894895cefffa9" exitCode=0 Jul 07 15:01:04.704168 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.704136 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" Jul 07 15:01:04.704392 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.704139 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" event={"ID":"ae40b01e-3ce0-427e-a7b2-9e8739f17165","Type":"ContainerDied","Data":"7b3d1b78e646e3fa2fcd73a0903957461ed9da124c8449b4e0a894895cefffa9"} Jul 07 15:01:04.704392 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.704257 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4" event={"ID":"ae40b01e-3ce0-427e-a7b2-9e8739f17165","Type":"ContainerDied","Data":"e2aa91f550948c7d9d9f15a2a6daeafb34f5232d50e055fcafbcbc1bcda1068a"} Jul 07 15:01:04.704392 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.704279 2555 scope.go:117] "RemoveContainer" containerID="7b3d1b78e646e3fa2fcd73a0903957461ed9da124c8449b4e0a894895cefffa9" Jul 07 15:01:04.713812 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.713796 2555 scope.go:117] "RemoveContainer" containerID="f6344a84ee33847d6096c46d8469a1ebca909c93e033ed8b15768c252717ebdf" Jul 07 15:01:04.721570 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.721554 2555 scope.go:117] "RemoveContainer" containerID="7b3d1b78e646e3fa2fcd73a0903957461ed9da124c8449b4e0a894895cefffa9" Jul 07 15:01:04.721827 ip-10-0-141-129 kubenswrapper[2555]: E0707 15:01:04.721808 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b3d1b78e646e3fa2fcd73a0903957461ed9da124c8449b4e0a894895cefffa9\": container with ID starting with 7b3d1b78e646e3fa2fcd73a0903957461ed9da124c8449b4e0a894895cefffa9 not found: ID does not exist" containerID="7b3d1b78e646e3fa2fcd73a0903957461ed9da124c8449b4e0a894895cefffa9" Jul 07 15:01:04.721879 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.721835 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b3d1b78e646e3fa2fcd73a0903957461ed9da124c8449b4e0a894895cefffa9"} err="failed to get container status \"7b3d1b78e646e3fa2fcd73a0903957461ed9da124c8449b4e0a894895cefffa9\": rpc error: code = NotFound desc = could not find container \"7b3d1b78e646e3fa2fcd73a0903957461ed9da124c8449b4e0a894895cefffa9\": container with ID starting with 7b3d1b78e646e3fa2fcd73a0903957461ed9da124c8449b4e0a894895cefffa9 not found: ID does not exist" Jul 07 15:01:04.721879 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.721852 2555 scope.go:117] "RemoveContainer" containerID="f6344a84ee33847d6096c46d8469a1ebca909c93e033ed8b15768c252717ebdf" Jul 07 15:01:04.722120 ip-10-0-141-129 kubenswrapper[2555]: E0707 15:01:04.722100 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6344a84ee33847d6096c46d8469a1ebca909c93e033ed8b15768c252717ebdf\": container with ID starting with f6344a84ee33847d6096c46d8469a1ebca909c93e033ed8b15768c252717ebdf not found: ID does not exist" containerID="f6344a84ee33847d6096c46d8469a1ebca909c93e033ed8b15768c252717ebdf" Jul 07 15:01:04.722187 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.722130 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6344a84ee33847d6096c46d8469a1ebca909c93e033ed8b15768c252717ebdf"} err="failed to get container status \"f6344a84ee33847d6096c46d8469a1ebca909c93e033ed8b15768c252717ebdf\": rpc error: code = NotFound desc = could not find container \"f6344a84ee33847d6096c46d8469a1ebca909c93e033ed8b15768c252717ebdf\": container with ID starting with f6344a84ee33847d6096c46d8469a1ebca909c93e033ed8b15768c252717ebdf not found: ID does not exist" Jul 07 15:01:04.739701 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.739673 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4"] Jul 07 15:01:04.750227 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:04.750200 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-54f2c7-openshift-default-799d56c59c-9jbf4"] Jul 07 15:01:05.181936 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:05.181899 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" path="/var/lib/kubelet/pods/ae40b01e-3ce0-427e-a7b2-9e8739f17165/volumes" Jul 07 15:01:08.060391 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.060361 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9"] Jul 07 15:01:08.060793 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.060761 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" containerName="istio-proxy" Jul 07 15:01:08.060793 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.060776 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" containerName="istio-proxy" Jul 07 15:01:08.060865 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.060845 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" containerName="istio-proxy" Jul 07 15:01:08.064015 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.063992 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.066743 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.066721 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-90fba0-openshift-default-dockercfg-6t5j6\"" Jul 07 15:01:08.085530 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.085497 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9"] Jul 07 15:01:08.142609 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.142567 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-workload-socket\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.142802 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.142615 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istiod-ca-cert\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.142802 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.142667 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-podinfo\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.142802 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.142708 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-workload-certs\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.142802 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.142758 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-data\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.143020 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.142808 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-credential-socket\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.143020 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.142879 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffrhr\" (UniqueName: \"kubernetes.io/projected/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-kube-api-access-ffrhr\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.143020 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.142956 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-token\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.143020 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.142993 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-envoy\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.243677 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.243644 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-workload-socket\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.243677 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.243684 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istiod-ca-cert\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.243910 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.243703 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-podinfo\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.243910 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.243725 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-workload-certs\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.243910 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.243770 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-data\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.243910 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.243789 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-credential-socket\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.243910 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.243848 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffrhr\" (UniqueName: \"kubernetes.io/projected/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-kube-api-access-ffrhr\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.243910 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.243906 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-token\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.244250 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.243940 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-envoy\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.244250 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.244126 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-workload-socket\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.244250 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.244225 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-data\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.244402 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.244257 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-workload-certs\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.244402 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.244363 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-credential-socket\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.244515 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.244493 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istiod-ca-cert\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.246450 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.246422 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-envoy\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.246575 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.246560 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-podinfo\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.254451 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.254430 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-token\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.254622 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.254606 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffrhr\" (UniqueName: \"kubernetes.io/projected/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-kube-api-access-ffrhr\") pod \"e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.374709 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.374626 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:08.563003 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.562979 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9"] Jul 07 15:01:08.565323 ip-10-0-141-129 kubenswrapper[2555]: W0707 15:01:08.565295 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dd7b9b2_a4e6_445f_a816_c7c90cfa8541.slice/crio-b4db2352ed3466694497eb89f6e7d774b57f8f5fca0f5fb83b327a66ab3e537d WatchSource:0}: Error finding container b4db2352ed3466694497eb89f6e7d774b57f8f5fca0f5fb83b327a66ab3e537d: Status 404 returned error can't find the container with id b4db2352ed3466694497eb89f6e7d774b57f8f5fca0f5fb83b327a66ab3e537d Jul 07 15:01:08.567388 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.567359 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:08.567456 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.567420 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:08.567456 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.567447 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:08.727111 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.727043 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" event={"ID":"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541","Type":"ContainerStarted","Data":"c98e2e61cee2bab74adcf0ab6f23984da19c3e8541937470c657a52ceff0915c"} Jul 07 15:01:08.727270 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.727138 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" event={"ID":"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541","Type":"ContainerStarted","Data":"b4db2352ed3466694497eb89f6e7d774b57f8f5fca0f5fb83b327a66ab3e537d"} Jul 07 15:01:08.753037 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:08.752978 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" podStartSLOduration=0.752960694 podStartE2EDuration="752.960694ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:08.750792047 +0000 UTC m=+2070.098845600" watchObservedRunningTime="2026-07-07 15:01:08.752960694 +0000 UTC m=+2070.101014246" Jul 07 15:01:09.376434 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:09.375821 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:09.391542 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:09.391501 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:09.732589 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:09.732560 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:09.733648 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:09.733629 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:14.769186 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:14.769150 2555 generic.go:356] "Generic (PLEG): container finished" podID="0c9ab688-6be2-4bce-8509-d874e98e272b" containerID="06d323ec9b80448d0a854bbc0d669fc26547d8eeb60c3de2b67f894fcdf25077" exitCode=0 Jul 07 15:01:14.769589 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:14.769224 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" event={"ID":"0c9ab688-6be2-4bce-8509-d874e98e272b","Type":"ContainerDied","Data":"06d323ec9b80448d0a854bbc0d669fc26547d8eeb60c3de2b67f894fcdf25077"} Jul 07 15:01:14.769632 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:14.769614 2555 scope.go:117] "RemoveContainer" containerID="06d323ec9b80448d0a854bbc0d669fc26547d8eeb60c3de2b67f894fcdf25077" Jul 07 15:01:14.770649 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:14.770623 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:14.770710 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:14.770694 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:14.770744 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:14.770725 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:15.777039 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:15.777004 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" event={"ID":"0c9ab688-6be2-4bce-8509-d874e98e272b","Type":"ContainerStarted","Data":"2792a324431c8500418df501a91320f3ac04db447e3a3ca0419cb6e2c88d062e"} Jul 07 15:01:16.258571 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:16.258532 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:16.259773 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:16.259742 2555 patch_prober.go:28] interesting pod/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 07 15:01:16.259900 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:16.259808 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" podUID="0c9ab688-6be2-4bce-8509-d874e98e272b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 07 15:01:16.688835 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:16.688728 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:17.259512 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:17.259473 2555 patch_prober.go:28] interesting pod/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 07 15:01:17.259910 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:17.259538 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" podUID="0c9ab688-6be2-4bce-8509-d874e98e272b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 07 15:01:18.259503 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:18.259463 2555 patch_prober.go:28] interesting pod/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 07 15:01:18.259691 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:18.259528 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" podUID="0c9ab688-6be2-4bce-8509-d874e98e272b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 07 15:01:19.259513 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:19.259472 2555 patch_prober.go:28] interesting pod/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 07 15:01:19.259693 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:19.259564 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" podUID="0c9ab688-6be2-4bce-8509-d874e98e272b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 07 15:01:20.263096 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:20.263043 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:20.263980 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:20.263962 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:22.828961 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:22.828930 2555 generic.go:356] "Generic (PLEG): container finished" podID="3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" containerID="c98e2e61cee2bab74adcf0ab6f23984da19c3e8541937470c657a52ceff0915c" exitCode=0 Jul 07 15:01:22.829374 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:22.828974 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" event={"ID":"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541","Type":"ContainerDied","Data":"c98e2e61cee2bab74adcf0ab6f23984da19c3e8541937470c657a52ceff0915c"} Jul 07 15:01:22.829374 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:22.829351 2555 scope.go:117] "RemoveContainer" containerID="c98e2e61cee2bab74adcf0ab6f23984da19c3e8541937470c657a52ceff0915c" Jul 07 15:01:22.830472 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:22.830445 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:22.830575 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:22.830511 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:22.830575 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:22.830543 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:23.835910 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:23.835873 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" event={"ID":"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541","Type":"ContainerStarted","Data":"6746e87133bc1729a6a8200b2bb39353589f34705c041ba3faf4e1cfe2e73d88"} Jul 07 15:01:24.375242 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:24.375200 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:24.376298 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:24.376271 2555 patch_prober.go:28] interesting pod/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 07 15:01:24.376525 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:24.376501 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" podUID="3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 07 15:01:24.732672 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:24.732636 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:25.375127 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:25.375053 2555 patch_prober.go:28] interesting pod/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 07 15:01:25.375599 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:25.375153 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" podUID="3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 07 15:01:26.376138 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:26.376099 2555 patch_prober.go:28] interesting pod/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 07 15:01:26.377136 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:26.376680 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" podUID="3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 07 15:01:27.375568 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:27.375517 2555 patch_prober.go:28] interesting pod/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 07 15:01:27.375751 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:27.375594 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" podUID="3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 07 15:01:28.380373 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:28.380341 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:28.381360 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:28.381343 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:31.436743 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:31.436712 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv"] Jul 07 15:01:31.437153 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:31.436906 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" podUID="0c9ab688-6be2-4bce-8509-d874e98e272b" containerName="istio-proxy" containerID="cri-o://2792a324431c8500418df501a91320f3ac04db447e3a3ca0419cb6e2c88d062e" gracePeriod=30 Jul 07 15:01:35.265677 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:35.265637 2555 patch_prober.go:28] interesting pod/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 15:01:35.266104 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:35.265707 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" podUID="0c9ab688-6be2-4bce-8509-d874e98e272b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 15:01:36.713465 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.713443 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:36.842241 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.842205 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-credential-socket\") pod \"0c9ab688-6be2-4bce-8509-d874e98e272b\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " Jul 07 15:01:36.842414 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.842261 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-workload-certs\") pod \"0c9ab688-6be2-4bce-8509-d874e98e272b\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " Jul 07 15:01:36.842414 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.842349 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-data\") pod \"0c9ab688-6be2-4bce-8509-d874e98e272b\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " Jul 07 15:01:36.842414 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.842377 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c9ab688-6be2-4bce-8509-d874e98e272b-istiod-ca-cert\") pod \"0c9ab688-6be2-4bce-8509-d874e98e272b\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " Jul 07 15:01:36.842546 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.842421 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hhm6\" (UniqueName: \"kubernetes.io/projected/0c9ab688-6be2-4bce-8509-d874e98e272b-kube-api-access-6hhm6\") pod \"0c9ab688-6be2-4bce-8509-d874e98e272b\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " Jul 07 15:01:36.842546 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.842445 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-envoy\") pod \"0c9ab688-6be2-4bce-8509-d874e98e272b\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " Jul 07 15:01:36.842546 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.842496 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-workload-socket\") pod \"0c9ab688-6be2-4bce-8509-d874e98e272b\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " Jul 07 15:01:36.842698 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.842535 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0c9ab688-6be2-4bce-8509-d874e98e272b" (UID: "0c9ab688-6be2-4bce-8509-d874e98e272b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:01:36.842698 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.842556 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-token\") pod \"0c9ab688-6be2-4bce-8509-d874e98e272b\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " Jul 07 15:01:36.842698 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.842632 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0c9ab688-6be2-4bce-8509-d874e98e272b" (UID: "0c9ab688-6be2-4bce-8509-d874e98e272b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:01:36.842698 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.842643 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-podinfo\") pod \"0c9ab688-6be2-4bce-8509-d874e98e272b\" (UID: \"0c9ab688-6be2-4bce-8509-d874e98e272b\") " Jul 07 15:01:36.842897 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.842803 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0c9ab688-6be2-4bce-8509-d874e98e272b" (UID: "0c9ab688-6be2-4bce-8509-d874e98e272b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:01:36.843039 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.842964 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-credential-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:36.843039 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.842998 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-workload-certs\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:36.843039 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.843015 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-workload-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:36.843516 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.843127 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c9ab688-6be2-4bce-8509-d874e98e272b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0c9ab688-6be2-4bce-8509-d874e98e272b" (UID: "0c9ab688-6be2-4bce-8509-d874e98e272b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:01:36.843794 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.843764 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-data" (OuterVolumeSpecName: "istio-data") pod "0c9ab688-6be2-4bce-8509-d874e98e272b" (UID: "0c9ab688-6be2-4bce-8509-d874e98e272b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:01:36.844891 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.844861 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-token" (OuterVolumeSpecName: "istio-token") pod "0c9ab688-6be2-4bce-8509-d874e98e272b" (UID: "0c9ab688-6be2-4bce-8509-d874e98e272b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:01:36.844891 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.844876 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0c9ab688-6be2-4bce-8509-d874e98e272b" (UID: "0c9ab688-6be2-4bce-8509-d874e98e272b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:01:36.844891 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.844880 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0c9ab688-6be2-4bce-8509-d874e98e272b" (UID: "0c9ab688-6be2-4bce-8509-d874e98e272b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:01:36.845038 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.844947 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c9ab688-6be2-4bce-8509-d874e98e272b-kube-api-access-6hhm6" (OuterVolumeSpecName: "kube-api-access-6hhm6") pod "0c9ab688-6be2-4bce-8509-d874e98e272b" (UID: "0c9ab688-6be2-4bce-8509-d874e98e272b"). InnerVolumeSpecName "kube-api-access-6hhm6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:01:36.901059 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.901025 2555 generic.go:356] "Generic (PLEG): container finished" podID="0c9ab688-6be2-4bce-8509-d874e98e272b" containerID="2792a324431c8500418df501a91320f3ac04db447e3a3ca0419cb6e2c88d062e" exitCode=0 Jul 07 15:01:36.901274 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.901110 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" Jul 07 15:01:36.901274 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.901108 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" event={"ID":"0c9ab688-6be2-4bce-8509-d874e98e272b","Type":"ContainerDied","Data":"2792a324431c8500418df501a91320f3ac04db447e3a3ca0419cb6e2c88d062e"} Jul 07 15:01:36.901274 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.901232 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv" event={"ID":"0c9ab688-6be2-4bce-8509-d874e98e272b","Type":"ContainerDied","Data":"7cafa7117bdf5ecf1fac149f8d63d16496b03e00c3edffc2ac75cb55f66b2234"} Jul 07 15:01:36.901274 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.901256 2555 scope.go:117] "RemoveContainer" containerID="2792a324431c8500418df501a91320f3ac04db447e3a3ca0419cb6e2c88d062e" Jul 07 15:01:36.910373 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.910356 2555 scope.go:117] "RemoveContainer" containerID="06d323ec9b80448d0a854bbc0d669fc26547d8eeb60c3de2b67f894fcdf25077" Jul 07 15:01:36.918115 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.918096 2555 scope.go:117] "RemoveContainer" containerID="2792a324431c8500418df501a91320f3ac04db447e3a3ca0419cb6e2c88d062e" Jul 07 15:01:36.918380 ip-10-0-141-129 kubenswrapper[2555]: E0707 15:01:36.918363 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2792a324431c8500418df501a91320f3ac04db447e3a3ca0419cb6e2c88d062e\": container with ID starting with 2792a324431c8500418df501a91320f3ac04db447e3a3ca0419cb6e2c88d062e not found: ID does not exist" containerID="2792a324431c8500418df501a91320f3ac04db447e3a3ca0419cb6e2c88d062e" Jul 07 15:01:36.918431 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.918389 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2792a324431c8500418df501a91320f3ac04db447e3a3ca0419cb6e2c88d062e"} err="failed to get container status \"2792a324431c8500418df501a91320f3ac04db447e3a3ca0419cb6e2c88d062e\": rpc error: code = NotFound desc = could not find container \"2792a324431c8500418df501a91320f3ac04db447e3a3ca0419cb6e2c88d062e\": container with ID starting with 2792a324431c8500418df501a91320f3ac04db447e3a3ca0419cb6e2c88d062e not found: ID does not exist" Jul 07 15:01:36.918431 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.918416 2555 scope.go:117] "RemoveContainer" containerID="06d323ec9b80448d0a854bbc0d669fc26547d8eeb60c3de2b67f894fcdf25077" Jul 07 15:01:36.918649 ip-10-0-141-129 kubenswrapper[2555]: E0707 15:01:36.918632 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06d323ec9b80448d0a854bbc0d669fc26547d8eeb60c3de2b67f894fcdf25077\": container with ID starting with 06d323ec9b80448d0a854bbc0d669fc26547d8eeb60c3de2b67f894fcdf25077 not found: ID does not exist" containerID="06d323ec9b80448d0a854bbc0d669fc26547d8eeb60c3de2b67f894fcdf25077" Jul 07 15:01:36.918705 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.918655 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06d323ec9b80448d0a854bbc0d669fc26547d8eeb60c3de2b67f894fcdf25077"} err="failed to get container status \"06d323ec9b80448d0a854bbc0d669fc26547d8eeb60c3de2b67f894fcdf25077\": rpc error: code = NotFound desc = could not find container \"06d323ec9b80448d0a854bbc0d669fc26547d8eeb60c3de2b67f894fcdf25077\": container with ID starting with 06d323ec9b80448d0a854bbc0d669fc26547d8eeb60c3de2b67f894fcdf25077 not found: ID does not exist" Jul 07 15:01:36.924891 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.924866 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv"] Jul 07 15:01:36.931209 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.931188 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-18bb73-openshift-default-6b6fc5b8bf-nzghv"] Jul 07 15:01:36.944385 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.944361 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-token\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:36.944385 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.944380 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-podinfo\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:36.944385 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.944388 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-data\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:36.944549 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.944397 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c9ab688-6be2-4bce-8509-d874e98e272b-istiod-ca-cert\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:36.944549 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.944405 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hhm6\" (UniqueName: \"kubernetes.io/projected/0c9ab688-6be2-4bce-8509-d874e98e272b-kube-api-access-6hhm6\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:36.944549 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:36.944414 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c9ab688-6be2-4bce-8509-d874e98e272b-istio-envoy\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:37.180649 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:37.180614 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c9ab688-6be2-4bce-8509-d874e98e272b" path="/var/lib/kubelet/pods/0c9ab688-6be2-4bce-8509-d874e98e272b/volumes" Jul 07 15:01:39.216914 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:39.216883 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 15:01:39.221471 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:39.221446 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 15:01:39.319104 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:39.319048 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9"] Jul 07 15:01:39.319339 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:39.319299 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" podUID="3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" containerName="istio-proxy" containerID="cri-o://6746e87133bc1729a6a8200b2bb39353589f34705c041ba3faf4e1cfe2e73d88" gracePeriod=30 Jul 07 15:01:40.227280 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.227244 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn"] Jul 07 15:01:40.227728 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.227597 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c9ab688-6be2-4bce-8509-d874e98e272b" containerName="istio-proxy" Jul 07 15:01:40.227728 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.227608 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c9ab688-6be2-4bce-8509-d874e98e272b" containerName="istio-proxy" Jul 07 15:01:40.227728 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.227617 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c9ab688-6be2-4bce-8509-d874e98e272b" containerName="istio-proxy" Jul 07 15:01:40.227728 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.227622 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c9ab688-6be2-4bce-8509-d874e98e272b" containerName="istio-proxy" Jul 07 15:01:40.227728 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.227641 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" containerName="istio-proxy" Jul 07 15:01:40.227728 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.227646 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" containerName="istio-proxy" Jul 07 15:01:40.227728 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.227702 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c9ab688-6be2-4bce-8509-d874e98e272b" containerName="istio-proxy" Jul 07 15:01:40.227728 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.227710 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c9ab688-6be2-4bce-8509-d874e98e272b" containerName="istio-proxy" Jul 07 15:01:40.227728 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.227718 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae40b01e-3ce0-427e-a7b2-9e8739f17165" containerName="istio-proxy" Jul 07 15:01:40.230840 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.230815 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.232819 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.232795 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-26fa43-openshift-default-dockercfg-9ncd5\"" Jul 07 15:01:40.246103 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.246080 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn"] Jul 07 15:01:40.377558 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.377522 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-workload-socket\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.377720 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.377597 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-envoy\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.377720 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.377665 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istiod-ca-cert\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.377839 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.377719 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-workload-certs\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.377839 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.377782 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkng2\" (UniqueName: \"kubernetes.io/projected/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-kube-api-access-jkng2\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.377839 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.377831 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-token\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.378012 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.377854 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-podinfo\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.378012 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.377873 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-credential-socket\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.378012 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.377930 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-data\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.478963 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.478867 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-envoy\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.478963 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.478919 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istiod-ca-cert\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.478963 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.478958 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-workload-certs\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.479268 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.478995 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkng2\" (UniqueName: \"kubernetes.io/projected/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-kube-api-access-jkng2\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.479268 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.479028 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-token\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.479268 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.479060 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-podinfo\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.479268 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.479113 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-credential-socket\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.479473 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.479267 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-data\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.479473 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.479331 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-workload-socket\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.479473 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.479418 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-workload-certs\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.479722 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.479699 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-data\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.479780 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.479726 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istiod-ca-cert\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.479780 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.479731 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-workload-socket\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.479780 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.479767 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-credential-socket\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.481772 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.481744 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-envoy\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.481880 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.481857 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-podinfo\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.487536 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.487511 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-token\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.487645 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.487577 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkng2\" (UniqueName: \"kubernetes.io/projected/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-kube-api-access-jkng2\") pod \"e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.542265 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.542239 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:40.725753 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.725716 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn"] Jul 07 15:01:40.739102 ip-10-0-141-129 kubenswrapper[2555]: W0707 15:01:40.739044 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0993ab4_a83a_4ed3_926b_efbc64dd8d90.slice/crio-6f0863fd4eb769f4217bd8b9568d6e66b5614f71d91963641a75b8b3b83d0891 WatchSource:0}: Error finding container 6f0863fd4eb769f4217bd8b9568d6e66b5614f71d91963641a75b8b3b83d0891: Status 404 returned error can't find the container with id 6f0863fd4eb769f4217bd8b9568d6e66b5614f71d91963641a75b8b3b83d0891 Jul 07 15:01:40.741433 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.741391 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:40.741504 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.741485 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:40.741582 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.741524 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:40.923773 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.923729 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" event={"ID":"e0993ab4-a83a-4ed3-926b-efbc64dd8d90","Type":"ContainerStarted","Data":"866d77b920b2877c889595a98bc27f1669d117d76dba0044f540ca742e1e1e8d"} Jul 07 15:01:40.923949 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.923792 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" event={"ID":"e0993ab4-a83a-4ed3-926b-efbc64dd8d90","Type":"ContainerStarted","Data":"6f0863fd4eb769f4217bd8b9568d6e66b5614f71d91963641a75b8b3b83d0891"} Jul 07 15:01:40.947432 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:40.946746 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" podStartSLOduration=0.946727459 podStartE2EDuration="946.727459ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:40.944731851 +0000 UTC m=+2102.292785403" watchObservedRunningTime="2026-07-07 15:01:40.946727459 +0000 UTC m=+2102.294781015" Jul 07 15:01:41.543264 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:41.543227 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:41.548014 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:41.547990 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:41.928827 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:41.928742 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:41.929651 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:41.929631 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:01:43.383397 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:43.383355 2555 patch_prober.go:28] interesting pod/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 15:01:43.383761 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:43.383426 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" podUID="3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 15:01:44.575976 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.575943 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:44.615021 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.614928 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-credential-socket\") pod \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " Jul 07 15:01:44.615021 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.614965 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffrhr\" (UniqueName: \"kubernetes.io/projected/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-kube-api-access-ffrhr\") pod \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " Jul 07 15:01:44.615021 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.614987 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-envoy\") pod \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " Jul 07 15:01:44.615370 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.615022 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-workload-socket\") pod \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " Jul 07 15:01:44.615370 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.615114 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-token\") pod \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " Jul 07 15:01:44.615370 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.615148 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-data\") pod \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " Jul 07 15:01:44.615370 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.615185 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-podinfo\") pod \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " Jul 07 15:01:44.615370 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.615217 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istiod-ca-cert\") pod \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " Jul 07 15:01:44.615370 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.615244 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-workload-certs\") pod \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\" (UID: \"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541\") " Jul 07 15:01:44.615370 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.615335 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" (UID: "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:01:44.615604 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.615532 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" (UID: "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:01:44.615652 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.615642 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-credential-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:44.615699 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.615658 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-workload-certs\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:44.615858 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.615782 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" (UID: "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:01:44.615994 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.615938 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" (UID: "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:01:44.616437 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.616386 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-data" (OuterVolumeSpecName: "istio-data") pod "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" (UID: "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:01:44.617685 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.617661 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" (UID: "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:01:44.618157 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.618107 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-kube-api-access-ffrhr" (OuterVolumeSpecName: "kube-api-access-ffrhr") pod "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" (UID: "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541"). InnerVolumeSpecName "kube-api-access-ffrhr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:01:44.618227 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.618147 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-token" (OuterVolumeSpecName: "istio-token") pod "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" (UID: "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:01:44.618275 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.618223 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" (UID: "3dd7b9b2-a4e6-445f-a816-c7c90cfa8541"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:01:44.716595 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.716558 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-podinfo\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:44.716595 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.716588 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istiod-ca-cert\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:44.716595 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.716603 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffrhr\" (UniqueName: \"kubernetes.io/projected/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-kube-api-access-ffrhr\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:44.716830 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.716615 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-envoy\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:44.716830 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.716629 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-workload-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:44.716830 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.716641 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-token\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:44.716830 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.716652 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541-istio-data\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:01:44.949116 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.949007 2555 generic.go:356] "Generic (PLEG): container finished" podID="3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" containerID="6746e87133bc1729a6a8200b2bb39353589f34705c041ba3faf4e1cfe2e73d88" exitCode=0 Jul 07 15:01:44.949116 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.949055 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" Jul 07 15:01:44.949116 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.949058 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" event={"ID":"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541","Type":"ContainerDied","Data":"6746e87133bc1729a6a8200b2bb39353589f34705c041ba3faf4e1cfe2e73d88"} Jul 07 15:01:44.949345 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.949126 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9" event={"ID":"3dd7b9b2-a4e6-445f-a816-c7c90cfa8541","Type":"ContainerDied","Data":"b4db2352ed3466694497eb89f6e7d774b57f8f5fca0f5fb83b327a66ab3e537d"} Jul 07 15:01:44.949345 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.949148 2555 scope.go:117] "RemoveContainer" containerID="6746e87133bc1729a6a8200b2bb39353589f34705c041ba3faf4e1cfe2e73d88" Jul 07 15:01:44.958298 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.958281 2555 scope.go:117] "RemoveContainer" containerID="c98e2e61cee2bab74adcf0ab6f23984da19c3e8541937470c657a52ceff0915c" Jul 07 15:01:44.966047 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.966026 2555 scope.go:117] "RemoveContainer" containerID="6746e87133bc1729a6a8200b2bb39353589f34705c041ba3faf4e1cfe2e73d88" Jul 07 15:01:44.966308 ip-10-0-141-129 kubenswrapper[2555]: E0707 15:01:44.966288 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6746e87133bc1729a6a8200b2bb39353589f34705c041ba3faf4e1cfe2e73d88\": container with ID starting with 6746e87133bc1729a6a8200b2bb39353589f34705c041ba3faf4e1cfe2e73d88 not found: ID does not exist" containerID="6746e87133bc1729a6a8200b2bb39353589f34705c041ba3faf4e1cfe2e73d88" Jul 07 15:01:44.966354 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.966316 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6746e87133bc1729a6a8200b2bb39353589f34705c041ba3faf4e1cfe2e73d88"} err="failed to get container status \"6746e87133bc1729a6a8200b2bb39353589f34705c041ba3faf4e1cfe2e73d88\": rpc error: code = NotFound desc = could not find container \"6746e87133bc1729a6a8200b2bb39353589f34705c041ba3faf4e1cfe2e73d88\": container with ID starting with 6746e87133bc1729a6a8200b2bb39353589f34705c041ba3faf4e1cfe2e73d88 not found: ID does not exist" Jul 07 15:01:44.966354 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.966334 2555 scope.go:117] "RemoveContainer" containerID="c98e2e61cee2bab74adcf0ab6f23984da19c3e8541937470c657a52ceff0915c" Jul 07 15:01:44.966547 ip-10-0-141-129 kubenswrapper[2555]: E0707 15:01:44.966530 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c98e2e61cee2bab74adcf0ab6f23984da19c3e8541937470c657a52ceff0915c\": container with ID starting with c98e2e61cee2bab74adcf0ab6f23984da19c3e8541937470c657a52ceff0915c not found: ID does not exist" containerID="c98e2e61cee2bab74adcf0ab6f23984da19c3e8541937470c657a52ceff0915c" Jul 07 15:01:44.966587 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.966550 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c98e2e61cee2bab74adcf0ab6f23984da19c3e8541937470c657a52ceff0915c"} err="failed to get container status \"c98e2e61cee2bab74adcf0ab6f23984da19c3e8541937470c657a52ceff0915c\": rpc error: code = NotFound desc = could not find container \"c98e2e61cee2bab74adcf0ab6f23984da19c3e8541937470c657a52ceff0915c\": container with ID starting with c98e2e61cee2bab74adcf0ab6f23984da19c3e8541937470c657a52ceff0915c not found: ID does not exist" Jul 07 15:01:44.971123 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.971102 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9"] Jul 07 15:01:44.976012 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:44.975988 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-90fba0-openshift-default-5588df78fc-h9rd9"] Jul 07 15:01:45.181889 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:45.181855 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" path="/var/lib/kubelet/pods/3dd7b9b2-a4e6-445f-a816-c7c90cfa8541/volumes" Jul 07 15:01:48.146940 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.146907 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8"] Jul 07 15:01:48.147504 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.147487 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" containerName="istio-proxy" Jul 07 15:01:48.147552 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.147508 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" containerName="istio-proxy" Jul 07 15:01:48.147552 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.147530 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" containerName="istio-proxy" Jul 07 15:01:48.147552 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.147538 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" containerName="istio-proxy" Jul 07 15:01:48.147654 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.147617 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" containerName="istio-proxy" Jul 07 15:01:48.147654 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.147627 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dd7b9b2-a4e6-445f-a816-c7c90cfa8541" containerName="istio-proxy" Jul 07 15:01:48.151191 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.151154 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.153639 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.153621 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-2f8e54-openshift-default-dockercfg-xjm56\"" Jul 07 15:01:48.164668 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.164643 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8"] Jul 07 15:01:48.249273 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.249238 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8caad990-61a2-4ae7-839d-0083ddfca690-istio-podinfo\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.249273 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.249279 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-istio-envoy\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.249506 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.249360 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfj2c\" (UniqueName: \"kubernetes.io/projected/8caad990-61a2-4ae7-839d-0083ddfca690-kube-api-access-bfj2c\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.249506 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.249405 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-workload-socket\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.249506 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.249464 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-credential-socket\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.249506 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.249490 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-workload-certs\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.249676 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.249515 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-istio-data\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.249676 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.249537 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8caad990-61a2-4ae7-839d-0083ddfca690-istiod-ca-cert\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.249676 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.249577 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8caad990-61a2-4ae7-839d-0083ddfca690-istio-token\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.350086 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.350037 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-istio-data\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.350269 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.350103 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8caad990-61a2-4ae7-839d-0083ddfca690-istiod-ca-cert\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.350269 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.350138 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8caad990-61a2-4ae7-839d-0083ddfca690-istio-token\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.350269 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.350163 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8caad990-61a2-4ae7-839d-0083ddfca690-istio-podinfo\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.350269 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.350193 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-istio-envoy\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.350269 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.350269 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfj2c\" (UniqueName: \"kubernetes.io/projected/8caad990-61a2-4ae7-839d-0083ddfca690-kube-api-access-bfj2c\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.350556 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.350307 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-workload-socket\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.350556 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.350373 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-istio-data\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.350556 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.350385 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-credential-socket\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.350556 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.350428 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-workload-certs\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.350770 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.350739 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-workload-certs\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.350770 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.350744 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-workload-socket\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.350879 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.350819 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-credential-socket\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.350879 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.350855 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8caad990-61a2-4ae7-839d-0083ddfca690-istiod-ca-cert\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.352555 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.352532 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-istio-envoy\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.352791 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.352773 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8caad990-61a2-4ae7-839d-0083ddfca690-istio-podinfo\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.358502 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.358470 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8caad990-61a2-4ae7-839d-0083ddfca690-istio-token\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.359104 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.359088 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfj2c\" (UniqueName: \"kubernetes.io/projected/8caad990-61a2-4ae7-839d-0083ddfca690-kube-api-access-bfj2c\") pod \"e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.461140 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.461047 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:48.599390 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.599350 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8"] Jul 07 15:01:48.602729 ip-10-0-141-129 kubenswrapper[2555]: W0707 15:01:48.602701 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8caad990_61a2_4ae7_839d_0083ddfca690.slice/crio-0cdf02005c5fe9429117ef64ffe3f1f34458e842a6d221293eb6995d13177bde WatchSource:0}: Error finding container 0cdf02005c5fe9429117ef64ffe3f1f34458e842a6d221293eb6995d13177bde: Status 404 returned error can't find the container with id 0cdf02005c5fe9429117ef64ffe3f1f34458e842a6d221293eb6995d13177bde Jul 07 15:01:48.604899 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.604868 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:48.604986 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.604929 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:48.604986 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.604958 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:48.971750 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.971714 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" event={"ID":"8caad990-61a2-4ae7-839d-0083ddfca690","Type":"ContainerStarted","Data":"1d4e8b9c998e6183af7aace32253d94af9d0b72b42e80826ce14a6471ef50a8f"} Jul 07 15:01:48.971954 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.971773 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" event={"ID":"8caad990-61a2-4ae7-839d-0083ddfca690","Type":"ContainerStarted","Data":"0cdf02005c5fe9429117ef64ffe3f1f34458e842a6d221293eb6995d13177bde"} Jul 07 15:01:48.994177 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:48.994128 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" podStartSLOduration=0.994111993 podStartE2EDuration="994.111993ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:48.991291742 +0000 UTC m=+2110.339345296" watchObservedRunningTime="2026-07-07 15:01:48.994111993 +0000 UTC m=+2110.342165546" Jul 07 15:01:49.461931 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:49.461845 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:49.463362 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:49.463334 2555 patch_prober.go:28] interesting pod/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 07 15:01:49.463481 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:49.463381 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" podUID="8caad990-61a2-4ae7-839d-0083ddfca690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 07 15:01:50.466230 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:50.466202 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:50.983943 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:50.983913 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:50.984883 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:50.984858 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:01:57.971226 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:57.971178 2555 patch_prober.go:28] interesting pod/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 15:01:57.971816 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:57.971241 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" podUID="e0993ab4-a83a-4ed3-926b-efbc64dd8d90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 15:01:59.031094 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:59.030984 2555 generic.go:356] "Generic (PLEG): container finished" podID="e0993ab4-a83a-4ed3-926b-efbc64dd8d90" containerID="866d77b920b2877c889595a98bc27f1669d117d76dba0044f540ca742e1e1e8d" exitCode=0 Jul 07 15:01:59.031094 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:59.031021 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" event={"ID":"e0993ab4-a83a-4ed3-926b-efbc64dd8d90","Type":"ContainerDied","Data":"866d77b920b2877c889595a98bc27f1669d117d76dba0044f540ca742e1e1e8d"} Jul 07 15:01:59.031511 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:59.031495 2555 scope.go:117] "RemoveContainer" containerID="866d77b920b2877c889595a98bc27f1669d117d76dba0044f540ca742e1e1e8d" Jul 07 15:01:59.032611 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:59.032582 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:59.032700 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:59.032684 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:01:59.032737 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:01:59.032724 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:02:00.038143 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:00.038112 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" event={"ID":"e0993ab4-a83a-4ed3-926b-efbc64dd8d90","Type":"ContainerStarted","Data":"60bf9921f36e459efc28948258fcd4c3b76c25f4839cdef914c9f0c20d08af1b"} Jul 07 15:02:00.542448 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:00.542406 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:02:00.542643 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:00.542617 2555 patch_prober.go:28] interesting pod/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 07 15:02:00.542701 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:00.542679 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" podUID="e0993ab4-a83a-4ed3-926b-efbc64dd8d90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 07 15:02:01.543112 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:01.543054 2555 patch_prober.go:28] interesting pod/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 07 15:02:01.543514 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:01.543138 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" podUID="e0993ab4-a83a-4ed3-926b-efbc64dd8d90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 07 15:02:02.543490 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:02.543455 2555 patch_prober.go:28] interesting pod/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 07 15:02:02.543930 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:02.543506 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" podUID="e0993ab4-a83a-4ed3-926b-efbc64dd8d90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 07 15:02:03.543187 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:03.543131 2555 patch_prober.go:28] interesting pod/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 07 15:02:03.543405 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:03.543205 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" podUID="e0993ab4-a83a-4ed3-926b-efbc64dd8d90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 07 15:02:04.546004 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:04.545975 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:02:04.546349 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:04.546300 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:02:04.546918 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:04.546903 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:02:07.026251 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:07.026208 2555 patch_prober.go:28] interesting pod/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 15:02:07.026692 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:07.026324 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" podUID="8caad990-61a2-4ae7-839d-0083ddfca690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 15:02:08.083495 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:08.083457 2555 generic.go:356] "Generic (PLEG): container finished" podID="8caad990-61a2-4ae7-839d-0083ddfca690" containerID="1d4e8b9c998e6183af7aace32253d94af9d0b72b42e80826ce14a6471ef50a8f" exitCode=0 Jul 07 15:02:08.083909 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:08.083522 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" event={"ID":"8caad990-61a2-4ae7-839d-0083ddfca690","Type":"ContainerDied","Data":"1d4e8b9c998e6183af7aace32253d94af9d0b72b42e80826ce14a6471ef50a8f"} Jul 07 15:02:08.084133 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:08.084117 2555 scope.go:117] "RemoveContainer" containerID="1d4e8b9c998e6183af7aace32253d94af9d0b72b42e80826ce14a6471ef50a8f" Jul 07 15:02:08.085394 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:08.085365 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:02:08.085490 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:08.085441 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:02:08.085490 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:08.085473 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 07 15:02:09.093535 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:09.093489 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" event={"ID":"8caad990-61a2-4ae7-839d-0083ddfca690","Type":"ContainerStarted","Data":"ccc8ba80083965bef5457f380fed9981328c22eccca4d0c4166c5c9c3fa5e954"} Jul 07 15:02:09.462046 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:09.461703 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:02:09.462046 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:09.461924 2555 patch_prober.go:28] interesting pod/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 07 15:02:09.462046 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:09.461973 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" podUID="8caad990-61a2-4ae7-839d-0083ddfca690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 07 15:02:10.462213 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:10.462181 2555 patch_prober.go:28] interesting pod/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 07 15:02:10.462590 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:10.462233 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" podUID="8caad990-61a2-4ae7-839d-0083ddfca690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 07 15:02:11.462390 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:11.462352 2555 patch_prober.go:28] interesting pod/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 07 15:02:11.462782 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:11.462410 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" podUID="8caad990-61a2-4ae7-839d-0083ddfca690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 07 15:02:12.461974 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:12.461935 2555 patch_prober.go:28] interesting pod/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 07 15:02:12.462191 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:12.461993 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" podUID="8caad990-61a2-4ae7-839d-0083ddfca690" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 07 15:02:13.464571 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:13.464537 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:02:13.464947 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:13.464775 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:02:13.465551 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:02:13.465533 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:05:43.263270 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.263241 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7"] Jul 07 15:05:43.266577 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.266549 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.271178 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.271134 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-26fa43\"/\"rate-test-model-26fa43-kserve-self-signed-certs\"" Jul 07 15:05:43.271838 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.271814 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-26fa43\"/\"default-dockercfg-ggs4z\"" Jul 07 15:05:43.271945 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.271886 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-26fa43\"/\"kube-root-ca.crt\"" Jul 07 15:05:43.274653 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.274636 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-26fa43\"/\"openshift-service-ca.crt\"" Jul 07 15:05:43.282371 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.282347 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7"] Jul 07 15:05:43.304383 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.304352 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40d588d4-fa2f-46d4-9320-c789ccb5b42b-tls-certs\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.304554 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.304408 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-model-cache\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.304625 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.304553 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-home\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.304625 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.304599 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-dshm\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.304728 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.304661 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-tmp-dir\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.304728 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.304687 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7qvb\" (UniqueName: \"kubernetes.io/projected/40d588d4-fa2f-46d4-9320-c789ccb5b42b-kube-api-access-b7qvb\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.406202 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.406157 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40d588d4-fa2f-46d4-9320-c789ccb5b42b-tls-certs\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.406407 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.406219 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-model-cache\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.406407 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.406313 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-home\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.406407 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.406348 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-dshm\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.406407 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.406383 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-tmp-dir\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.406601 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.406408 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7qvb\" (UniqueName: \"kubernetes.io/projected/40d588d4-fa2f-46d4-9320-c789ccb5b42b-kube-api-access-b7qvb\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.406669 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.406644 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-model-cache\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.406735 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.406688 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-home\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.406790 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.406753 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-tmp-dir\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.408768 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.408741 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-dshm\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.409095 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.409049 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40d588d4-fa2f-46d4-9320-c789ccb5b42b-tls-certs\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.416272 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.416250 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7qvb\" (UniqueName: \"kubernetes.io/projected/40d588d4-fa2f-46d4-9320-c789ccb5b42b-kube-api-access-b7qvb\") pod \"rate-test-model-26fa43-kserve-6d78bdd687-78sg7\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.580022 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.579925 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:43.725670 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:43.725635 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7"] Jul 07 15:05:43.727851 ip-10-0-141-129 kubenswrapper[2555]: W0707 15:05:43.727823 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40d588d4_fa2f_46d4_9320_c789ccb5b42b.slice/crio-9fe1132a74c0452e1b90f72aae0d2713e322d353313da5043b9c9a3d853a842b WatchSource:0}: Error finding container 9fe1132a74c0452e1b90f72aae0d2713e322d353313da5043b9c9a3d853a842b: Status 404 returned error can't find the container with id 9fe1132a74c0452e1b90f72aae0d2713e322d353313da5043b9c9a3d853a842b Jul 07 15:05:44.126998 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:44.126957 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" event={"ID":"40d588d4-fa2f-46d4-9320-c789ccb5b42b","Type":"ContainerStarted","Data":"58af7b5971d1e848f7e69dced9edb5da2c5a4afafa7a895aa57bc756dddb9a42"} Jul 07 15:05:44.126998 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:44.127003 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" event={"ID":"40d588d4-fa2f-46d4-9320-c789ccb5b42b","Type":"ContainerStarted","Data":"9fe1132a74c0452e1b90f72aae0d2713e322d353313da5043b9c9a3d853a842b"} Jul 07 15:05:44.152779 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:44.152726 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" podStartSLOduration=1.152708569 podStartE2EDuration="1.152708569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:44.151792537 +0000 UTC m=+2345.499846086" watchObservedRunningTime="2026-07-07 15:05:44.152708569 +0000 UTC m=+2345.500762121" Jul 07 15:05:53.581056 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:53.581012 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:53.581532 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:53.581092 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:53.593616 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:53.593593 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:05:54.187321 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:05:54.187292 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:06:39.247409 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:06:39.247380 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 15:06:39.252847 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:06:39.252825 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ml5j_86594764-14f9-43fb-a595-518d0c910187/ovn-acl-logging/0.log" Jul 07 15:07:10.126150 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:10.126110 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7"] Jul 07 15:07:25.146841 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.146777 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" podUID="40d588d4-fa2f-46d4-9320-c789ccb5b42b" containerName="main" containerID="cri-o://58af7b5971d1e848f7e69dced9edb5da2c5a4afafa7a895aa57bc756dddb9a42" gracePeriod=45 Jul 07 15:07:25.399891 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.399834 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:07:25.590672 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.590638 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-tmp-dir\") pod \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " Jul 07 15:07:25.590852 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.590695 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-model-cache\") pod \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " Jul 07 15:07:25.590852 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.590722 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7qvb\" (UniqueName: \"kubernetes.io/projected/40d588d4-fa2f-46d4-9320-c789ccb5b42b-kube-api-access-b7qvb\") pod \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " Jul 07 15:07:25.590852 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.590757 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40d588d4-fa2f-46d4-9320-c789ccb5b42b-tls-certs\") pod \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " Jul 07 15:07:25.590852 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.590776 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-dshm\") pod \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " Jul 07 15:07:25.591108 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.590857 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-home\") pod \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\" (UID: \"40d588d4-fa2f-46d4-9320-c789ccb5b42b\") " Jul 07 15:07:25.591179 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.591111 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-model-cache" (OuterVolumeSpecName: "model-cache") pod "40d588d4-fa2f-46d4-9320-c789ccb5b42b" (UID: "40d588d4-fa2f-46d4-9320-c789ccb5b42b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:25.591245 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.591212 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-home" (OuterVolumeSpecName: "home") pod "40d588d4-fa2f-46d4-9320-c789ccb5b42b" (UID: "40d588d4-fa2f-46d4-9320-c789ccb5b42b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:25.591693 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.591321 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-model-cache\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:07:25.591693 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.591364 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-home\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:07:25.591693 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.591536 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "40d588d4-fa2f-46d4-9320-c789ccb5b42b" (UID: "40d588d4-fa2f-46d4-9320-c789ccb5b42b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:25.593290 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.593266 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40d588d4-fa2f-46d4-9320-c789ccb5b42b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "40d588d4-fa2f-46d4-9320-c789ccb5b42b" (UID: "40d588d4-fa2f-46d4-9320-c789ccb5b42b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:07:25.593290 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.593285 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40d588d4-fa2f-46d4-9320-c789ccb5b42b-kube-api-access-b7qvb" (OuterVolumeSpecName: "kube-api-access-b7qvb") pod "40d588d4-fa2f-46d4-9320-c789ccb5b42b" (UID: "40d588d4-fa2f-46d4-9320-c789ccb5b42b"). InnerVolumeSpecName "kube-api-access-b7qvb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:07:25.593429 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.593358 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-dshm" (OuterVolumeSpecName: "dshm") pod "40d588d4-fa2f-46d4-9320-c789ccb5b42b" (UID: "40d588d4-fa2f-46d4-9320-c789ccb5b42b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:25.618153 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.618125 2555 generic.go:356] "Generic (PLEG): container finished" podID="40d588d4-fa2f-46d4-9320-c789ccb5b42b" containerID="58af7b5971d1e848f7e69dced9edb5da2c5a4afafa7a895aa57bc756dddb9a42" exitCode=0 Jul 07 15:07:25.618245 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.618174 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" Jul 07 15:07:25.618245 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.618204 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" event={"ID":"40d588d4-fa2f-46d4-9320-c789ccb5b42b","Type":"ContainerDied","Data":"58af7b5971d1e848f7e69dced9edb5da2c5a4afafa7a895aa57bc756dddb9a42"} Jul 07 15:07:25.618245 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.618238 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7" event={"ID":"40d588d4-fa2f-46d4-9320-c789ccb5b42b","Type":"ContainerDied","Data":"9fe1132a74c0452e1b90f72aae0d2713e322d353313da5043b9c9a3d853a842b"} Jul 07 15:07:25.618387 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.618255 2555 scope.go:117] "RemoveContainer" containerID="58af7b5971d1e848f7e69dced9edb5da2c5a4afafa7a895aa57bc756dddb9a42" Jul 07 15:07:25.627035 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.627020 2555 scope.go:117] "RemoveContainer" containerID="58af7b5971d1e848f7e69dced9edb5da2c5a4afafa7a895aa57bc756dddb9a42" Jul 07 15:07:25.627328 ip-10-0-141-129 kubenswrapper[2555]: E0707 15:07:25.627311 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58af7b5971d1e848f7e69dced9edb5da2c5a4afafa7a895aa57bc756dddb9a42\": container with ID starting with 58af7b5971d1e848f7e69dced9edb5da2c5a4afafa7a895aa57bc756dddb9a42 not found: ID does not exist" containerID="58af7b5971d1e848f7e69dced9edb5da2c5a4afafa7a895aa57bc756dddb9a42" Jul 07 15:07:25.627398 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.627339 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58af7b5971d1e848f7e69dced9edb5da2c5a4afafa7a895aa57bc756dddb9a42"} err="failed to get container status \"58af7b5971d1e848f7e69dced9edb5da2c5a4afafa7a895aa57bc756dddb9a42\": rpc error: code = NotFound desc = could not find container \"58af7b5971d1e848f7e69dced9edb5da2c5a4afafa7a895aa57bc756dddb9a42\": container with ID starting with 58af7b5971d1e848f7e69dced9edb5da2c5a4afafa7a895aa57bc756dddb9a42 not found: ID does not exist" Jul 07 15:07:25.637756 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.637727 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7"] Jul 07 15:07:25.641632 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.641612 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/rate-test-model-26fa43-kserve-6d78bdd687-78sg7"] Jul 07 15:07:25.692543 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.692475 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-tmp-dir\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:07:25.692543 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.692505 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7qvb\" (UniqueName: \"kubernetes.io/projected/40d588d4-fa2f-46d4-9320-c789ccb5b42b-kube-api-access-b7qvb\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:07:25.692543 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.692514 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40d588d4-fa2f-46d4-9320-c789ccb5b42b-tls-certs\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:07:25.692543 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:25.692524 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40d588d4-fa2f-46d4-9320-c789ccb5b42b-dshm\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:07:27.180792 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:27.180755 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40d588d4-fa2f-46d4-9320-c789ccb5b42b" path="/var/lib/kubelet/pods/40d588d4-fa2f-46d4-9320-c789ccb5b42b/volumes" Jul 07 15:07:47.681864 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:47.681823 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn"] Jul 07 15:07:47.682414 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:47.682134 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" podUID="e0993ab4-a83a-4ed3-926b-efbc64dd8d90" containerName="istio-proxy" containerID="cri-o://60bf9921f36e459efc28948258fcd4c3b76c25f4839cdef914c9f0c20d08af1b" gracePeriod=30 Jul 07 15:07:49.548809 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:49.548771 2555 patch_prober.go:28] interesting pod/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 15:07:49.549223 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:49.548849 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" podUID="e0993ab4-a83a-4ed3-926b-efbc64dd8d90" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 15:07:52.934447 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:52.934421 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:07:53.048443 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.048410 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-podinfo\") pod \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " Jul 07 15:07:53.048609 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.048475 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-data\") pod \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " Jul 07 15:07:53.048609 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.048498 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-workload-socket\") pod \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " Jul 07 15:07:53.048609 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.048526 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-workload-certs\") pod \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " Jul 07 15:07:53.048609 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.048541 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkng2\" (UniqueName: \"kubernetes.io/projected/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-kube-api-access-jkng2\") pod \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " Jul 07 15:07:53.048609 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.048583 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-envoy\") pod \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " Jul 07 15:07:53.048861 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.048616 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-credential-socket\") pod \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " Jul 07 15:07:53.048861 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.048652 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istiod-ca-cert\") pod \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " Jul 07 15:07:53.048861 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.048688 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-token\") pod \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\" (UID: \"e0993ab4-a83a-4ed3-926b-efbc64dd8d90\") " Jul 07 15:07:53.048861 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.048810 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e0993ab4-a83a-4ed3-926b-efbc64dd8d90" (UID: "e0993ab4-a83a-4ed3-926b-efbc64dd8d90"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:53.049106 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.049016 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e0993ab4-a83a-4ed3-926b-efbc64dd8d90" (UID: "e0993ab4-a83a-4ed3-926b-efbc64dd8d90"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:53.049106 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.049039 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-workload-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:07:53.049219 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.049117 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e0993ab4-a83a-4ed3-926b-efbc64dd8d90" (UID: "e0993ab4-a83a-4ed3-926b-efbc64dd8d90"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:53.049538 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.049513 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e0993ab4-a83a-4ed3-926b-efbc64dd8d90" (UID: "e0993ab4-a83a-4ed3-926b-efbc64dd8d90"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:07:53.049798 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.049769 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-data" (OuterVolumeSpecName: "istio-data") pod "e0993ab4-a83a-4ed3-926b-efbc64dd8d90" (UID: "e0993ab4-a83a-4ed3-926b-efbc64dd8d90"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:53.051014 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.050991 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e0993ab4-a83a-4ed3-926b-efbc64dd8d90" (UID: "e0993ab4-a83a-4ed3-926b-efbc64dd8d90"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:53.051014 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.051006 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-token" (OuterVolumeSpecName: "istio-token") pod "e0993ab4-a83a-4ed3-926b-efbc64dd8d90" (UID: "e0993ab4-a83a-4ed3-926b-efbc64dd8d90"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:07:53.051167 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.051028 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e0993ab4-a83a-4ed3-926b-efbc64dd8d90" (UID: "e0993ab4-a83a-4ed3-926b-efbc64dd8d90"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:07:53.051167 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.051103 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-kube-api-access-jkng2" (OuterVolumeSpecName: "kube-api-access-jkng2") pod "e0993ab4-a83a-4ed3-926b-efbc64dd8d90" (UID: "e0993ab4-a83a-4ed3-926b-efbc64dd8d90"). InnerVolumeSpecName "kube-api-access-jkng2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:07:53.150418 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.150388 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-data\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:07:53.150418 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.150413 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-workload-certs\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:07:53.150418 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.150422 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkng2\" (UniqueName: \"kubernetes.io/projected/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-kube-api-access-jkng2\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:07:53.150627 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.150429 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-envoy\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:07:53.150627 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.150438 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-credential-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:07:53.150627 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.150445 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istiod-ca-cert\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:07:53.150627 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.150454 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-token\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:07:53.150627 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.150461 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0993ab4-a83a-4ed3-926b-efbc64dd8d90-istio-podinfo\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:07:53.760328 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.760291 2555 generic.go:356] "Generic (PLEG): container finished" podID="e0993ab4-a83a-4ed3-926b-efbc64dd8d90" containerID="60bf9921f36e459efc28948258fcd4c3b76c25f4839cdef914c9f0c20d08af1b" exitCode=0 Jul 07 15:07:53.760537 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.760338 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" Jul 07 15:07:53.760537 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.760347 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" event={"ID":"e0993ab4-a83a-4ed3-926b-efbc64dd8d90","Type":"ContainerDied","Data":"60bf9921f36e459efc28948258fcd4c3b76c25f4839cdef914c9f0c20d08af1b"} Jul 07 15:07:53.760537 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.760398 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn" event={"ID":"e0993ab4-a83a-4ed3-926b-efbc64dd8d90","Type":"ContainerDied","Data":"6f0863fd4eb769f4217bd8b9568d6e66b5614f71d91963641a75b8b3b83d0891"} Jul 07 15:07:53.760537 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.760425 2555 scope.go:117] "RemoveContainer" containerID="60bf9921f36e459efc28948258fcd4c3b76c25f4839cdef914c9f0c20d08af1b" Jul 07 15:07:53.769399 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.769381 2555 scope.go:117] "RemoveContainer" containerID="866d77b920b2877c889595a98bc27f1669d117d76dba0044f540ca742e1e1e8d" Jul 07 15:07:53.777903 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.777881 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn"] Jul 07 15:07:53.778483 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.778468 2555 scope.go:117] "RemoveContainer" containerID="60bf9921f36e459efc28948258fcd4c3b76c25f4839cdef914c9f0c20d08af1b" Jul 07 15:07:53.778726 ip-10-0-141-129 kubenswrapper[2555]: E0707 15:07:53.778699 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60bf9921f36e459efc28948258fcd4c3b76c25f4839cdef914c9f0c20d08af1b\": container with ID starting with 60bf9921f36e459efc28948258fcd4c3b76c25f4839cdef914c9f0c20d08af1b not found: ID does not exist" containerID="60bf9921f36e459efc28948258fcd4c3b76c25f4839cdef914c9f0c20d08af1b" Jul 07 15:07:53.778781 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.778736 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60bf9921f36e459efc28948258fcd4c3b76c25f4839cdef914c9f0c20d08af1b"} err="failed to get container status \"60bf9921f36e459efc28948258fcd4c3b76c25f4839cdef914c9f0c20d08af1b\": rpc error: code = NotFound desc = could not find container \"60bf9921f36e459efc28948258fcd4c3b76c25f4839cdef914c9f0c20d08af1b\": container with ID starting with 60bf9921f36e459efc28948258fcd4c3b76c25f4839cdef914c9f0c20d08af1b not found: ID does not exist" Jul 07 15:07:53.778781 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.778753 2555 scope.go:117] "RemoveContainer" containerID="866d77b920b2877c889595a98bc27f1669d117d76dba0044f540ca742e1e1e8d" Jul 07 15:07:53.779044 ip-10-0-141-129 kubenswrapper[2555]: E0707 15:07:53.779025 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"866d77b920b2877c889595a98bc27f1669d117d76dba0044f540ca742e1e1e8d\": container with ID starting with 866d77b920b2877c889595a98bc27f1669d117d76dba0044f540ca742e1e1e8d not found: ID does not exist" containerID="866d77b920b2877c889595a98bc27f1669d117d76dba0044f540ca742e1e1e8d" Jul 07 15:07:53.779109 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.779050 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"866d77b920b2877c889595a98bc27f1669d117d76dba0044f540ca742e1e1e8d"} err="failed to get container status \"866d77b920b2877c889595a98bc27f1669d117d76dba0044f540ca742e1e1e8d\": rpc error: code = NotFound desc = could not find container \"866d77b920b2877c889595a98bc27f1669d117d76dba0044f540ca742e1e1e8d\": container with ID starting with 866d77b920b2877c889595a98bc27f1669d117d76dba0044f540ca742e1e1e8d not found: ID does not exist" Jul 07 15:07:53.783966 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:53.783946 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-26fa43-openshift-default-7cf7ddd77f-dv5vn"] Jul 07 15:07:55.181264 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:55.181231 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0993ab4-a83a-4ed3-926b-efbc64dd8d90" path="/var/lib/kubelet/pods/e0993ab4-a83a-4ed3-926b-efbc64dd8d90/volumes" Jul 07 15:07:55.759854 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:55.759826 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8"] Jul 07 15:07:55.760134 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:55.760108 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" podUID="8caad990-61a2-4ae7-839d-0083ddfca690" containerName="istio-proxy" containerID="cri-o://ccc8ba80083965bef5457f380fed9981328c22eccca4d0c4166c5c9c3fa5e954" gracePeriod=30 Jul 07 15:07:58.467059 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:58.467021 2555 patch_prober.go:28] interesting pod/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 15:07:58.467435 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:07:58.467100 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" podUID="8caad990-61a2-4ae7-839d-0083ddfca690" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 15:08:01.019767 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.019716 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:08:01.132050 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.132018 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-workload-socket\") pod \"8caad990-61a2-4ae7-839d-0083ddfca690\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " Jul 07 15:08:01.132209 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.132095 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8caad990-61a2-4ae7-839d-0083ddfca690-istio-token\") pod \"8caad990-61a2-4ae7-839d-0083ddfca690\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " Jul 07 15:08:01.132209 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.132156 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-istio-data\") pod \"8caad990-61a2-4ae7-839d-0083ddfca690\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " Jul 07 15:08:01.132209 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.132180 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-istio-envoy\") pod \"8caad990-61a2-4ae7-839d-0083ddfca690\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " Jul 07 15:08:01.132373 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.132211 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8caad990-61a2-4ae7-839d-0083ddfca690-istiod-ca-cert\") pod \"8caad990-61a2-4ae7-839d-0083ddfca690\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " Jul 07 15:08:01.132373 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.132232 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfj2c\" (UniqueName: \"kubernetes.io/projected/8caad990-61a2-4ae7-839d-0083ddfca690-kube-api-access-bfj2c\") pod \"8caad990-61a2-4ae7-839d-0083ddfca690\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " Jul 07 15:08:01.132373 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.132253 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-credential-socket\") pod \"8caad990-61a2-4ae7-839d-0083ddfca690\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " Jul 07 15:08:01.132373 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.132268 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-workload-certs\") pod \"8caad990-61a2-4ae7-839d-0083ddfca690\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " Jul 07 15:08:01.132373 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.132316 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8caad990-61a2-4ae7-839d-0083ddfca690-istio-podinfo\") pod \"8caad990-61a2-4ae7-839d-0083ddfca690\" (UID: \"8caad990-61a2-4ae7-839d-0083ddfca690\") " Jul 07 15:08:01.132373 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.132313 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8caad990-61a2-4ae7-839d-0083ddfca690" (UID: "8caad990-61a2-4ae7-839d-0083ddfca690"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:08:01.132683 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.132594 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-workload-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:08:01.132683 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.132599 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8caad990-61a2-4ae7-839d-0083ddfca690" (UID: "8caad990-61a2-4ae7-839d-0083ddfca690"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:08:01.132790 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.132678 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8caad990-61a2-4ae7-839d-0083ddfca690-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8caad990-61a2-4ae7-839d-0083ddfca690" (UID: "8caad990-61a2-4ae7-839d-0083ddfca690"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:08:01.132846 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.132823 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8caad990-61a2-4ae7-839d-0083ddfca690" (UID: "8caad990-61a2-4ae7-839d-0083ddfca690"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:08:01.133034 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.133012 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-istio-data" (OuterVolumeSpecName: "istio-data") pod "8caad990-61a2-4ae7-839d-0083ddfca690" (UID: "8caad990-61a2-4ae7-839d-0083ddfca690"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:08:01.134506 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.134483 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8caad990-61a2-4ae7-839d-0083ddfca690-istio-token" (OuterVolumeSpecName: "istio-token") pod "8caad990-61a2-4ae7-839d-0083ddfca690" (UID: "8caad990-61a2-4ae7-839d-0083ddfca690"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:08:01.134640 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.134619 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8caad990-61a2-4ae7-839d-0083ddfca690-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8caad990-61a2-4ae7-839d-0083ddfca690" (UID: "8caad990-61a2-4ae7-839d-0083ddfca690"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:08:01.134921 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.134896 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8caad990-61a2-4ae7-839d-0083ddfca690" (UID: "8caad990-61a2-4ae7-839d-0083ddfca690"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:08:01.134921 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.134906 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8caad990-61a2-4ae7-839d-0083ddfca690-kube-api-access-bfj2c" (OuterVolumeSpecName: "kube-api-access-bfj2c") pod "8caad990-61a2-4ae7-839d-0083ddfca690" (UID: "8caad990-61a2-4ae7-839d-0083ddfca690"). InnerVolumeSpecName "kube-api-access-bfj2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:08:01.233697 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.233669 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8caad990-61a2-4ae7-839d-0083ddfca690-istiod-ca-cert\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:08:01.233841 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.233716 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfj2c\" (UniqueName: \"kubernetes.io/projected/8caad990-61a2-4ae7-839d-0083ddfca690-kube-api-access-bfj2c\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:08:01.233841 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.233731 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-credential-socket\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:08:01.233841 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.233742 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-workload-certs\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:08:01.233841 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.233755 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8caad990-61a2-4ae7-839d-0083ddfca690-istio-podinfo\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:08:01.233841 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.233766 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8caad990-61a2-4ae7-839d-0083ddfca690-istio-token\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:08:01.233841 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.233777 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-istio-data\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:08:01.233841 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.233788 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8caad990-61a2-4ae7-839d-0083ddfca690-istio-envoy\") on node \"ip-10-0-141-129.ec2.internal\" DevicePath \"\"" Jul 07 15:08:01.528563 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.528527 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4s65s_9ef112f3-0a1b-4e2f-b925-79e9455ea0c4/manager/0.log" Jul 07 15:08:01.658412 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.658385 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xbzf4_2929b9cf-77eb-43d9-aa9a-7f115af06b0c/manager/0.log" Jul 07 15:08:01.808814 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.808728 2555 generic.go:356] "Generic (PLEG): container finished" podID="8caad990-61a2-4ae7-839d-0083ddfca690" containerID="ccc8ba80083965bef5457f380fed9981328c22eccca4d0c4166c5c9c3fa5e954" exitCode=0 Jul 07 15:08:01.808814 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.808790 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" Jul 07 15:08:01.809044 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.808818 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" event={"ID":"8caad990-61a2-4ae7-839d-0083ddfca690","Type":"ContainerDied","Data":"ccc8ba80083965bef5457f380fed9981328c22eccca4d0c4166c5c9c3fa5e954"} Jul 07 15:08:01.809044 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.808906 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8" event={"ID":"8caad990-61a2-4ae7-839d-0083ddfca690","Type":"ContainerDied","Data":"0cdf02005c5fe9429117ef64ffe3f1f34458e842a6d221293eb6995d13177bde"} Jul 07 15:08:01.809044 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.808937 2555 scope.go:117] "RemoveContainer" containerID="ccc8ba80083965bef5457f380fed9981328c22eccca4d0c4166c5c9c3fa5e954" Jul 07 15:08:01.817425 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.817411 2555 scope.go:117] "RemoveContainer" containerID="1d4e8b9c998e6183af7aace32253d94af9d0b72b42e80826ce14a6471ef50a8f" Jul 07 15:08:01.825203 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.825155 2555 scope.go:117] "RemoveContainer" containerID="ccc8ba80083965bef5457f380fed9981328c22eccca4d0c4166c5c9c3fa5e954" Jul 07 15:08:01.825629 ip-10-0-141-129 kubenswrapper[2555]: E0707 15:08:01.825546 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccc8ba80083965bef5457f380fed9981328c22eccca4d0c4166c5c9c3fa5e954\": container with ID starting with ccc8ba80083965bef5457f380fed9981328c22eccca4d0c4166c5c9c3fa5e954 not found: ID does not exist" containerID="ccc8ba80083965bef5457f380fed9981328c22eccca4d0c4166c5c9c3fa5e954" Jul 07 15:08:01.825629 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.825584 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc8ba80083965bef5457f380fed9981328c22eccca4d0c4166c5c9c3fa5e954"} err="failed to get container status \"ccc8ba80083965bef5457f380fed9981328c22eccca4d0c4166c5c9c3fa5e954\": rpc error: code = NotFound desc = could not find container \"ccc8ba80083965bef5457f380fed9981328c22eccca4d0c4166c5c9c3fa5e954\": container with ID starting with ccc8ba80083965bef5457f380fed9981328c22eccca4d0c4166c5c9c3fa5e954 not found: ID does not exist" Jul 07 15:08:01.825896 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.825631 2555 scope.go:117] "RemoveContainer" containerID="1d4e8b9c998e6183af7aace32253d94af9d0b72b42e80826ce14a6471ef50a8f" Jul 07 15:08:01.826175 ip-10-0-141-129 kubenswrapper[2555]: E0707 15:08:01.826049 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d4e8b9c998e6183af7aace32253d94af9d0b72b42e80826ce14a6471ef50a8f\": container with ID starting with 1d4e8b9c998e6183af7aace32253d94af9d0b72b42e80826ce14a6471ef50a8f not found: ID does not exist" containerID="1d4e8b9c998e6183af7aace32253d94af9d0b72b42e80826ce14a6471ef50a8f" Jul 07 15:08:01.826175 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.826164 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d4e8b9c998e6183af7aace32253d94af9d0b72b42e80826ce14a6471ef50a8f"} err="failed to get container status \"1d4e8b9c998e6183af7aace32253d94af9d0b72b42e80826ce14a6471ef50a8f\": rpc error: code = NotFound desc = could not find container \"1d4e8b9c998e6183af7aace32253d94af9d0b72b42e80826ce14a6471ef50a8f\": container with ID starting with 1d4e8b9c998e6183af7aace32253d94af9d0b72b42e80826ce14a6471ef50a8f not found: ID does not exist" Jul 07 15:08:01.827449 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.827431 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8"] Jul 07 15:08:01.834350 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:01.834330 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-2f8e54-openshift-default-6dfc49895-6wwl8"] Jul 07 15:08:02.710199 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:02.710171 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7868f6cbb6-mcp4w_67b654c2-802f-47eb-b6cd-dcb657b48043/copy-manifests/0.log" Jul 07 15:08:02.718059 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:02.718032 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7868f6cbb6-mcp4w_67b654c2-802f-47eb-b6cd-dcb657b48043/manager/0.log" Jul 07 15:08:03.180227 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:03.180197 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8caad990-61a2-4ae7-839d-0083ddfca690" path="/var/lib/kubelet/pods/8caad990-61a2-4ae7-839d-0083ddfca690/volumes" Jul 07 15:08:04.605023 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:04.604992 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-psxch_91d482f7-422e-459d-a537-0349e791fa0e/manager/0.log" Jul 07 15:08:04.971050 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:04.971024 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-fpwq8_657af0d6-c76a-414c-a1bf-3a2546643bab/manager/0.log" Jul 07 15:08:05.227171 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:05.227088 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-mj22f_2c0d0e79-7cf7-4605-9a72-7823389f4e6d/manager/0.log" Jul 07 15:08:05.581714 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:05.581611 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6787575j_9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21/istio-proxy/0.log" Jul 07 15:08:06.905846 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:06.905806 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-lj6n7_0e2ab784-7f67-4e0c-bb1d-e433bd9133cb/main/0.log" Jul 07 15:08:07.133674 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:07.133647 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbp4hq7_86b5da0d-c202-46e1-a258-87b8ef53002c/main/0.log" Jul 07 15:08:14.321993 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:14.321947 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8c9rl_cb20bfdc-dd4a-4961-a3ce-07939d8cd7c6/global-pull-secret-syncer/0.log" Jul 07 15:08:14.559896 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:14.559864 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sc8gn_c055b3b3-1fac-45d3-82c1-a3e18bb068f8/konnectivity-agent/0.log" Jul 07 15:08:14.718928 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:14.718902 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-129.ec2.internal_4a862d27acdd6adcd1a30a65484081bd/haproxy/0.log" Jul 07 15:08:19.936332 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:19.936258 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-psxch_91d482f7-422e-459d-a537-0349e791fa0e/manager/0.log" Jul 07 15:08:20.192741 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:20.192641 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-fpwq8_657af0d6-c76a-414c-a1bf-3a2546643bab/manager/0.log" Jul 07 15:08:20.307688 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:20.307655 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-mj22f_2c0d0e79-7cf7-4605-9a72-7823389f4e6d/manager/0.log" Jul 07 15:08:21.973046 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:21.973019 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e28fbf92-8bc8-44c1-84f2-e90cb01d91f3/alertmanager/0.log" Jul 07 15:08:22.003104 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.003063 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e28fbf92-8bc8-44c1-84f2-e90cb01d91f3/config-reloader/0.log" Jul 07 15:08:22.031769 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.031749 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e28fbf92-8bc8-44c1-84f2-e90cb01d91f3/kube-rbac-proxy-web/0.log" Jul 07 15:08:22.070018 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.069992 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e28fbf92-8bc8-44c1-84f2-e90cb01d91f3/kube-rbac-proxy/0.log" Jul 07 15:08:22.100490 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.100471 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e28fbf92-8bc8-44c1-84f2-e90cb01d91f3/kube-rbac-proxy-metric/0.log" Jul 07 15:08:22.129134 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.129104 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e28fbf92-8bc8-44c1-84f2-e90cb01d91f3/prom-label-proxy/0.log" Jul 07 15:08:22.157488 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.157468 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e28fbf92-8bc8-44c1-84f2-e90cb01d91f3/init-config-reloader/0.log" Jul 07 15:08:22.593883 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.593856 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rzvgc_fffeadca-9677-487f-a937-65985edbba4d/node-exporter/0.log" Jul 07 15:08:22.619088 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.619044 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rzvgc_fffeadca-9677-487f-a937-65985edbba4d/kube-rbac-proxy/0.log" Jul 07 15:08:22.647243 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.647222 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rzvgc_fffeadca-9677-487f-a937-65985edbba4d/init-textfile/0.log" Jul 07 15:08:22.770615 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.770589 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1361e560-3fa4-43f8-96fb-23eb94bb9ce4/prometheus/0.log" Jul 07 15:08:22.795842 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.795823 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1361e560-3fa4-43f8-96fb-23eb94bb9ce4/config-reloader/0.log" Jul 07 15:08:22.821850 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.821825 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1361e560-3fa4-43f8-96fb-23eb94bb9ce4/thanos-sidecar/0.log" Jul 07 15:08:22.848231 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.848212 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1361e560-3fa4-43f8-96fb-23eb94bb9ce4/kube-rbac-proxy-web/0.log" Jul 07 15:08:22.877619 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.877596 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1361e560-3fa4-43f8-96fb-23eb94bb9ce4/kube-rbac-proxy/0.log" Jul 07 15:08:22.904290 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.904272 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1361e560-3fa4-43f8-96fb-23eb94bb9ce4/kube-rbac-proxy-thanos/0.log" Jul 07 15:08:22.930836 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:22.930814 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1361e560-3fa4-43f8-96fb-23eb94bb9ce4/init-config-reloader/0.log" Jul 07 15:08:23.175194 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:23.175118 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65fdbf6b89-sfgth_b13d5133-cbd7-4d28-8f88-ea22a4a14014/thanos-query/0.log" Jul 07 15:08:23.204329 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:23.204305 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65fdbf6b89-sfgth_b13d5133-cbd7-4d28-8f88-ea22a4a14014/kube-rbac-proxy-web/0.log" Jul 07 15:08:23.231547 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:23.231524 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65fdbf6b89-sfgth_b13d5133-cbd7-4d28-8f88-ea22a4a14014/kube-rbac-proxy/0.log" Jul 07 15:08:23.259590 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:23.259562 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65fdbf6b89-sfgth_b13d5133-cbd7-4d28-8f88-ea22a4a14014/prom-label-proxy/0.log" Jul 07 15:08:23.289751 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:23.289723 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65fdbf6b89-sfgth_b13d5133-cbd7-4d28-8f88-ea22a4a14014/kube-rbac-proxy-rules/0.log" Jul 07 15:08:23.314957 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:23.314936 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65fdbf6b89-sfgth_b13d5133-cbd7-4d28-8f88-ea22a4a14014/kube-rbac-proxy-metrics/0.log" Jul 07 15:08:24.023603 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.023570 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n"] Jul 07 15:08:24.023947 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.023932 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8caad990-61a2-4ae7-839d-0083ddfca690" containerName="istio-proxy" Jul 07 15:08:24.023947 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.023946 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="8caad990-61a2-4ae7-839d-0083ddfca690" containerName="istio-proxy" Jul 07 15:08:24.024108 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.023957 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8caad990-61a2-4ae7-839d-0083ddfca690" containerName="istio-proxy" Jul 07 15:08:24.024108 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.023963 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="8caad990-61a2-4ae7-839d-0083ddfca690" containerName="istio-proxy" Jul 07 15:08:24.024108 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.023978 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0993ab4-a83a-4ed3-926b-efbc64dd8d90" containerName="istio-proxy" Jul 07 15:08:24.024108 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.023983 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0993ab4-a83a-4ed3-926b-efbc64dd8d90" containerName="istio-proxy" Jul 07 15:08:24.024108 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.023994 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40d588d4-fa2f-46d4-9320-c789ccb5b42b" containerName="main" Jul 07 15:08:24.024108 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.024000 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="40d588d4-fa2f-46d4-9320-c789ccb5b42b" containerName="main" Jul 07 15:08:24.024108 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.024011 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0993ab4-a83a-4ed3-926b-efbc64dd8d90" containerName="istio-proxy" Jul 07 15:08:24.024108 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.024016 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0993ab4-a83a-4ed3-926b-efbc64dd8d90" containerName="istio-proxy" Jul 07 15:08:24.024108 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.024082 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="8caad990-61a2-4ae7-839d-0083ddfca690" containerName="istio-proxy" Jul 07 15:08:24.024108 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.024095 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0993ab4-a83a-4ed3-926b-efbc64dd8d90" containerName="istio-proxy" Jul 07 15:08:24.024108 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.024101 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="40d588d4-fa2f-46d4-9320-c789ccb5b42b" containerName="main" Jul 07 15:08:24.026043 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.026026 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.028230 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.028202 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-btsff\"/\"kube-root-ca.crt\"" Jul 07 15:08:24.028327 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.028202 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-btsff\"/\"openshift-service-ca.crt\"" Jul 07 15:08:24.028691 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.028676 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-btsff\"/\"default-dockercfg-kcwx6\"" Jul 07 15:08:24.041696 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.041673 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n"] Jul 07 15:08:24.138712 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.138675 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkz9j\" (UniqueName: \"kubernetes.io/projected/39b4b8d0-0be9-4518-98fb-db42072ad107-kube-api-access-tkz9j\") pod \"perf-node-gather-daemonset-dfm2n\" (UID: \"39b4b8d0-0be9-4518-98fb-db42072ad107\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.138712 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.138712 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39b4b8d0-0be9-4518-98fb-db42072ad107-sys\") pod \"perf-node-gather-daemonset-dfm2n\" (UID: \"39b4b8d0-0be9-4518-98fb-db42072ad107\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.138927 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.138735 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/39b4b8d0-0be9-4518-98fb-db42072ad107-podres\") pod \"perf-node-gather-daemonset-dfm2n\" (UID: \"39b4b8d0-0be9-4518-98fb-db42072ad107\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.138927 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.138818 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/39b4b8d0-0be9-4518-98fb-db42072ad107-proc\") pod \"perf-node-gather-daemonset-dfm2n\" (UID: \"39b4b8d0-0be9-4518-98fb-db42072ad107\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.138927 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.138855 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39b4b8d0-0be9-4518-98fb-db42072ad107-lib-modules\") pod \"perf-node-gather-daemonset-dfm2n\" (UID: \"39b4b8d0-0be9-4518-98fb-db42072ad107\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.239448 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.239409 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkz9j\" (UniqueName: \"kubernetes.io/projected/39b4b8d0-0be9-4518-98fb-db42072ad107-kube-api-access-tkz9j\") pod \"perf-node-gather-daemonset-dfm2n\" (UID: \"39b4b8d0-0be9-4518-98fb-db42072ad107\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.239448 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.239452 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39b4b8d0-0be9-4518-98fb-db42072ad107-sys\") pod \"perf-node-gather-daemonset-dfm2n\" (UID: \"39b4b8d0-0be9-4518-98fb-db42072ad107\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.240003 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.239487 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/39b4b8d0-0be9-4518-98fb-db42072ad107-podres\") pod \"perf-node-gather-daemonset-dfm2n\" (UID: \"39b4b8d0-0be9-4518-98fb-db42072ad107\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.240003 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.239517 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/39b4b8d0-0be9-4518-98fb-db42072ad107-proc\") pod \"perf-node-gather-daemonset-dfm2n\" (UID: \"39b4b8d0-0be9-4518-98fb-db42072ad107\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.240003 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.239555 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39b4b8d0-0be9-4518-98fb-db42072ad107-lib-modules\") pod \"perf-node-gather-daemonset-dfm2n\" (UID: \"39b4b8d0-0be9-4518-98fb-db42072ad107\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.240003 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.239603 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39b4b8d0-0be9-4518-98fb-db42072ad107-sys\") pod \"perf-node-gather-daemonset-dfm2n\" (UID: \"39b4b8d0-0be9-4518-98fb-db42072ad107\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.240003 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.239613 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/39b4b8d0-0be9-4518-98fb-db42072ad107-proc\") pod \"perf-node-gather-daemonset-dfm2n\" (UID: \"39b4b8d0-0be9-4518-98fb-db42072ad107\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.240003 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.239662 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/39b4b8d0-0be9-4518-98fb-db42072ad107-podres\") pod \"perf-node-gather-daemonset-dfm2n\" (UID: \"39b4b8d0-0be9-4518-98fb-db42072ad107\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.240003 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.239695 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39b4b8d0-0be9-4518-98fb-db42072ad107-lib-modules\") pod \"perf-node-gather-daemonset-dfm2n\" (UID: \"39b4b8d0-0be9-4518-98fb-db42072ad107\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.249762 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.249739 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkz9j\" (UniqueName: \"kubernetes.io/projected/39b4b8d0-0be9-4518-98fb-db42072ad107-kube-api-access-tkz9j\") pod \"perf-node-gather-daemonset-dfm2n\" (UID: \"39b4b8d0-0be9-4518-98fb-db42072ad107\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.335499 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.335414 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.468803 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.468770 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n"] Jul 07 15:08:24.472111 ip-10-0-141-129 kubenswrapper[2555]: W0707 15:08:24.472084 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod39b4b8d0_0be9_4518_98fb_db42072ad107.slice/crio-10da0ba9dfb0680c371d5aac941f8a7cc9a6329a1394c7a70c853bf1bc9fd8ca WatchSource:0}: Error finding container 10da0ba9dfb0680c371d5aac941f8a7cc9a6329a1394c7a70c853bf1bc9fd8ca: Status 404 returned error can't find the container with id 10da0ba9dfb0680c371d5aac941f8a7cc9a6329a1394c7a70c853bf1bc9fd8ca Jul 07 15:08:24.916759 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.916727 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" event={"ID":"39b4b8d0-0be9-4518-98fb-db42072ad107","Type":"ContainerStarted","Data":"c437f254e60ab1ec88dcc096a7cf4a68120f69584ad507b727be1cd6b59b93a9"} Jul 07 15:08:24.916997 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.916777 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" event={"ID":"39b4b8d0-0be9-4518-98fb-db42072ad107","Type":"ContainerStarted","Data":"10da0ba9dfb0680c371d5aac941f8a7cc9a6329a1394c7a70c853bf1bc9fd8ca"} Jul 07 15:08:24.916997 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.916863 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:24.939183 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:24.939110 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" podStartSLOduration=0.939094999 podStartE2EDuration="939.094999ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:08:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:08:24.937427623 +0000 UTC m=+2506.285481172" watchObservedRunningTime="2026-07-07 15:08:24.939094999 +0000 UTC m=+2506.287148552" Jul 07 15:08:25.705085 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:25.705040 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-bbblk_b705da46-8252-44d5-ae93-2bb5f79b5b97/download-server/0.log" Jul 07 15:08:27.507972 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:27.507939 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b2slt_b4dfc0ec-e629-439f-b444-1a6784f4f29b/dns/0.log" Jul 07 15:08:27.536046 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:27.536015 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b2slt_b4dfc0ec-e629-439f-b444-1a6784f4f29b/kube-rbac-proxy/0.log" Jul 07 15:08:27.568142 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:27.568121 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dz5b9_fdcc1ba3-fd82-4fcb-be4b-92340a548edd/dns-node-resolver/0.log" Jul 07 15:08:28.181634 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:28.181604 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6ccd6cbfc5-7pvdb_a3571b67-a8e9-49f3-a6ab-cae8dd8bb559/registry/0.log" Jul 07 15:08:28.265554 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:28.265515 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mbx8j_f029d40b-288d-4026-97a5-119ff263ed93/node-ca/0.log" Jul 07 15:08:29.328960 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:29.328915 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6787575j_9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21/istio-proxy/0.log" Jul 07 15:08:30.562776 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:30.562752 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7r4qc_88fc15e7-95c6-453e-afa7-e9b3ae0384ec/serve-healthcheck-canary/0.log" Jul 07 15:08:30.931550 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:30.931464 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-dfm2n" Jul 07 15:08:31.199782 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:31.199704 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mjxmj_9413cf3e-94cd-4a72-9c07-495de8f1e33c/kube-rbac-proxy/0.log" Jul 07 15:08:31.227734 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:31.227700 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mjxmj_9413cf3e-94cd-4a72-9c07-495de8f1e33c/exporter/0.log" Jul 07 15:08:31.256097 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:31.256047 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mjxmj_9413cf3e-94cd-4a72-9c07-495de8f1e33c/extractor/0.log" Jul 07 15:08:33.901322 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:33.901240 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4s65s_9ef112f3-0a1b-4e2f-b925-79e9455ea0c4/manager/0.log" Jul 07 15:08:34.038240 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:34.038206 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xbzf4_2929b9cf-77eb-43d9-aa9a-7f115af06b0c/manager/0.log" Jul 07 15:08:34.507190 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:34.507161 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7868f6cbb6-mcp4w_67b654c2-802f-47eb-b6cd-dcb657b48043/manager/0.log" Jul 07 15:08:34.535089 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:34.535043 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7868f6cbb6-mcp4w_67b654c2-802f-47eb-b6cd-dcb657b48043/copy-manifests/0.log" Jul 07 15:08:34.756865 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:34.756826 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6787575j_9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21/istio-proxy/0.log" Jul 07 15:08:36.145835 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:36.145747 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-849574fc57-vrvsl_eaa03334-56df-42c4-9d42-05ac60413b2b/manager/0.log" Jul 07 15:08:36.477282 ip-10-0-141-129 kubenswrapper[2555]: I0707 15:08:36.477240 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6787575j_9720233d-2f4b-4f8e-b5f9-04ba9a8bcf21/istio-proxy/0.log"