Jul 14 00:59:30.968608 ip-10-0-143-159 systemd[1]: Starting Kubernetes Kubelet... Jul 14 00:59:31.425749 ip-10-0-143-159 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 00:59:31.425749 ip-10-0-143-159 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 00:59:31.425749 ip-10-0-143-159 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 00:59:31.425749 ip-10-0-143-159 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 00:59:31.425749 ip-10-0-143-159 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 00:59:31.426939 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.426792 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 00:59:31.432825 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432802 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 00:59:31.432825 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432819 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 00:59:31.432825 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432825 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 00:59:31.432825 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432829 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 00:59:31.432825 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432833 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432838 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432844 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432848 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432851 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432855 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432861 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432866 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432871 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432875 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432879 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432884 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432888 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432892 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432905 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432911 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432915 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432918 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432923 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 00:59:31.433172 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432927 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432931 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432935 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432939 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432943 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432947 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432951 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432955 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432959 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432963 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432967 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432972 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432976 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432981 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432985 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432991 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.432995 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433000 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433004 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433008 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 00:59:31.433811 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433013 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433017 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433022 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433026 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433030 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433035 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433039 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433043 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433048 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433052 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433056 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433060 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433064 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433068 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433073 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433077 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433081 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433085 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433090 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433096 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 00:59:31.434296 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433100 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433109 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433117 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433123 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433128 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433132 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433138 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433143 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433147 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433152 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433157 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433162 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433167 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433171 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433175 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433180 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433184 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433188 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433193 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 00:59:31.435169 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433197 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433202 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433207 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433212 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433216 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433221 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433225 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433229 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433236 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433241 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433245 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433250 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433255 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433260 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433264 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433268 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 00:59:31.435938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.433272 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 00:59:31.436645 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436626 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 00:59:31.436645 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436645 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436649 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436652 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436655 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436659 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436663 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436667 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436671 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436674 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436677 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436680 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436682 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436685 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436688 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436690 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436693 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436695 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436698 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436701 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 00:59:31.436698 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436704 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436706 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436709 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436712 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436715 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436718 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436720 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436722 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436725 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436727 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436730 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436732 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436738 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436741 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436744 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436746 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436748 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436751 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436754 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436757 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 00:59:31.437201 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436759 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436761 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436764 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436766 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436769 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436771 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436774 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436776 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436778 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436781 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436784 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436787 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436789 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436792 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436794 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436797 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436800 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436802 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436805 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436807 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 00:59:31.437737 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436810 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436812 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436815 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436817 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436819 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436822 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436824 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436827 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436829 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436832 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436835 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436837 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436839 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436842 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436844 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436850 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436854 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436857 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436859 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436862 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 00:59:31.438244 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436865 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436867 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436871 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436876 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436880 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436883 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436886 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436889 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436892 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436894 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436897 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436899 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436902 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436905 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436908 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436910 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436912 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436915 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.436918 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437338 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 00:59:31.438840 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437344 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437347 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437350 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437353 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437355 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437358 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437361 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437363 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437366 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437368 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437371 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437374 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437376 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437379 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437382 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437385 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437388 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437391 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437394 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437396 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 00:59:31.439349 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437399 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437401 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437404 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437406 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437409 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437411 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437414 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437418 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437422 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437424 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437427 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437429 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437432 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437434 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437437 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437439 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437442 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437444 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437447 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437449 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 00:59:31.439856 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437452 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437454 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437456 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437459 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437462 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437465 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437467 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437470 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437473 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437476 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437478 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437481 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437483 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437486 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437489 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437491 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437495 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437497 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437499 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437502 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 00:59:31.440360 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437504 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437507 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437510 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437512 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437515 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437517 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437519 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437522 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437524 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437527 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437531 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437534 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437537 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437540 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437542 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437545 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437548 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437551 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437554 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 00:59:31.440843 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437557 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437561 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437565 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437570 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437573 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437576 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437579 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437582 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437585 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437588 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437590 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437593 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437596 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437599 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437601 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437604 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437606 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437609 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.437611 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437694 2582 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 00:59:31.441299 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437702 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437710 2582 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437715 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437719 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437722 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437727 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437731 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437735 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437738 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437742 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437749 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437752 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437756 2582 flags.go:64] FLAG: --cgroup-root="" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437759 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437761 2582 flags.go:64] FLAG: --client-ca-file="" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437765 2582 flags.go:64] FLAG: --cloud-provider="external" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437768 2582 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437776 2582 flags.go:64] FLAG: --cluster-domain="" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437779 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437782 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437785 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437789 2582 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437793 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 00:59:31.441794 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437796 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437799 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437802 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437806 2582 flags.go:64] FLAG: --contention-profiling="false" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437809 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437812 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437815 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437818 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437822 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437825 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437828 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437831 2582 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437834 2582 flags.go:64] FLAG: --enable-server="true" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437837 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437841 2582 flags.go:64] FLAG: --event-burst="100" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437844 2582 flags.go:64] FLAG: --event-qps="50" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437847 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437850 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437853 2582 flags.go:64] FLAG: --eviction-hard="" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437857 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437860 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437863 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437866 2582 flags.go:64] FLAG: --eviction-soft="" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437869 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437872 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 00:59:31.442415 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437875 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437878 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437881 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437884 2582 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437886 2582 flags.go:64] FLAG: --feature-gates="" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437890 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437893 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437896 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437899 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437902 2582 flags.go:64] FLAG: --healthz-port="10248" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437905 2582 flags.go:64] FLAG: --help="false" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437907 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-143-159.ec2.internal" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437910 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437913 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437915 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437919 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437922 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437925 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437928 2582 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437931 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437933 2582 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437936 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437940 2582 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437943 2582 flags.go:64] FLAG: --kube-reserved="" Jul 14 00:59:31.443074 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437946 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437949 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437952 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437955 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437958 2582 flags.go:64] FLAG: --lock-file="" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437961 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437964 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437966 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437973 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437976 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437979 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437982 2582 flags.go:64] FLAG: --logging-format="text" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437985 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437988 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437991 2582 flags.go:64] FLAG: --manifest-url="" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437994 2582 flags.go:64] FLAG: --manifest-url-header="" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.437998 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438001 2582 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438005 2582 flags.go:64] FLAG: --max-pods="110" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438008 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438011 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438014 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438017 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438020 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438023 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 00:59:31.443697 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438026 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438033 2582 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438036 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438039 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438042 2582 flags.go:64] FLAG: --pod-cidr="" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438045 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438051 2582 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438054 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438057 2582 flags.go:64] FLAG: --pods-per-core="0" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438060 2582 flags.go:64] FLAG: --port="10250" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438063 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438066 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b6f049e8fecc4260" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438069 2582 flags.go:64] FLAG: --qos-reserved="" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438073 2582 flags.go:64] FLAG: --read-only-port="10255" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438075 2582 flags.go:64] FLAG: --register-node="true" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438078 2582 flags.go:64] FLAG: --register-with-taints="" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438082 2582 flags.go:64] FLAG: --registry-burst="10" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438085 2582 flags.go:64] FLAG: --registry-qps="5" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438088 2582 flags.go:64] FLAG: --reserved-cpus="" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438091 2582 flags.go:64] FLAG: --reserved-memory="" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438095 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438098 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438101 2582 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438104 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438107 2582 flags.go:64] FLAG: --runonce="false" Jul 14 00:59:31.444307 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438110 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438113 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438116 2582 flags.go:64] FLAG: --seccomp-default="false" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438119 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438121 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438126 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438129 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438132 2582 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438134 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438137 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438140 2582 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438143 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438146 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438149 2582 flags.go:64] FLAG: --system-cgroups="" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438152 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438157 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438160 2582 flags.go:64] FLAG: --tls-cert-file="" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438163 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438167 2582 flags.go:64] FLAG: --tls-min-version="" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438169 2582 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438172 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438175 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438178 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438181 2582 flags.go:64] FLAG: --v="2" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438185 2582 flags.go:64] FLAG: --version="false" Jul 14 00:59:31.444937 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438189 2582 flags.go:64] FLAG: --vmodule="" Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438193 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.438196 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438347 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438352 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438355 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438358 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438362 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438366 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438369 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438372 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438375 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438380 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438382 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438385 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438387 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438390 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438393 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438395 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 00:59:31.445578 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438400 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438402 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438405 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438410 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438414 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438419 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438423 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438428 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438432 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438434 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438437 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438440 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438442 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438445 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438448 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438451 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438454 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438456 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438459 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438461 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 00:59:31.446071 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438463 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438466 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438469 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438471 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438474 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438477 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438480 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438482 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438485 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438487 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438490 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438493 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438497 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438500 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438502 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438505 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438508 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438510 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438513 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 00:59:31.446604 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438515 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438517 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438520 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438522 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438525 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438528 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438530 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438532 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438535 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438538 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438540 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438543 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438546 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438548 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438550 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438553 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438555 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438558 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438561 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 00:59:31.447069 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438564 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438568 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438571 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438573 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438576 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438578 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438582 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438585 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438587 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438590 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438592 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438595 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438597 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438600 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438602 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438604 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438607 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438609 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438612 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438614 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 00:59:31.447558 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438617 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 00:59:31.448051 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438619 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 00:59:31.448051 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438622 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 00:59:31.448051 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438625 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 00:59:31.448051 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.438627 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 00:59:31.448051 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.439424 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 00:59:31.448051 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.446948 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 00:59:31.448051 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.446967 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 00:59:31.448051 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.447846 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 00:59:31.448051 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.447854 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 00:59:31.448051 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447916 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 00:59:31.448051 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447922 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 14 00:59:31.448051 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447925 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 00:59:31.448051 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447929 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 00:59:31.448051 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447932 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 00:59:31.448051 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447935 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447938 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447940 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447943 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447946 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447949 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447951 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447954 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447957 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447959 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447962 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447965 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447967 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447970 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447972 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447975 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447977 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447980 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447984 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 00:59:31.448442 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447989 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447992 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447994 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.447997 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448000 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448004 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448007 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448009 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448012 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448016 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448019 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448022 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448025 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448028 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448031 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448033 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448036 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448039 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448042 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448044 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 00:59:31.448910 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448047 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448049 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448052 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448054 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448057 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448060 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448063 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448065 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448067 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448070 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448073 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448075 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448079 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448083 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448088 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448090 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448093 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448096 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448099 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 00:59:31.449450 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448102 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448104 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448107 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448110 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448112 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448115 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448118 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448120 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448122 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448125 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448128 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448130 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448133 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448135 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448138 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448141 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448143 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448145 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448148 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448150 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 00:59:31.449913 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448153 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 00:59:31.450484 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448155 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 00:59:31.450484 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448157 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 00:59:31.450484 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448160 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 00:59:31.450484 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448162 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 00:59:31.450484 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448164 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 00:59:31.450484 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448167 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 00:59:31.450484 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448170 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 00:59:31.450484 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448173 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 00:59:31.450484 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448175 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 00:59:31.450484 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448178 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 00:59:31.450484 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448181 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 00:59:31.450484 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448184 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 00:59:31.450484 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448186 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 00:59:31.450484 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448189 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 00:59:31.450484 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448192 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 00:59:31.450848 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.448197 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 00:59:31.450848 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448348 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 00:59:31.450848 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448353 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 00:59:31.450848 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448356 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 00:59:31.450848 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448359 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 00:59:31.450848 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448362 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 00:59:31.450848 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448364 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 00:59:31.450848 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448367 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 00:59:31.450848 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448370 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 00:59:31.450848 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448372 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 00:59:31.450848 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448376 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 00:59:31.450848 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448380 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 00:59:31.450848 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448383 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 00:59:31.450848 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448386 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448389 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448392 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448394 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448397 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448400 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448403 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448405 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448407 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448410 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448412 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448415 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448418 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448421 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448424 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448426 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448429 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448431 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448434 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448436 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 00:59:31.451190 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448439 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448441 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448444 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448447 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448449 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448452 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448455 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448457 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448460 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448462 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448465 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448467 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448470 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448472 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448475 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448477 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448479 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448482 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448485 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448487 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 00:59:31.451700 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448489 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448492 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448495 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448498 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448500 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448503 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448506 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448510 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448513 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448516 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448519 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448522 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448524 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448527 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448530 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448532 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448535 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448537 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448540 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 00:59:31.452186 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448543 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448545 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448547 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448550 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448553 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448557 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448559 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448562 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448564 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448567 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448570 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448572 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448574 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448577 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448579 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448582 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448585 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448587 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448590 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448593 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 00:59:31.452658 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448596 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 00:59:31.453147 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448598 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 00:59:31.453147 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448601 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 00:59:31.453147 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448603 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 00:59:31.453147 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448605 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 00:59:31.453147 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448608 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 00:59:31.453147 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448613 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 00:59:31.453147 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:31.448615 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 00:59:31.453147 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.448620 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 00:59:31.453147 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.448746 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 00:59:31.453147 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.451394 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 00:59:31.453586 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.453154 2582 server.go:1019] "Starting client certificate rotation" Jul 14 00:59:31.453586 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.453255 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 00:59:31.453586 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.453295 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 00:59:31.456970 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.456953 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 00:59:31.458175 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.458162 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 00:59:31.474924 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.474890 2582 log.go:25] "Validated CRI v1 runtime API" Jul 14 00:59:31.477520 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.477496 2582 log.go:25] "Validated CRI v1 image API" Jul 14 00:59:31.479222 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.479206 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 00:59:31.484821 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.484798 2582 fs.go:135] Filesystem UUIDs: map[441be7f3-ea73-4c93-922f-fc35e679824d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 bf992478-1e39-4bf0-98b4-66b62f4a88f5:/dev/nvme0n1p3] Jul 14 00:59:31.484911 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.484820 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 00:59:31.486157 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.486137 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 00:59:31.491473 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.491354 2582 manager.go:216] Machine: {Timestamp:2026-07-14 00:59:31.48994372 +0000 UTC m=+0.392799212 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104788 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ef1c46dee3b9dd8cb41143897de33 SystemUUID:ec2ef1c4-6dee-3b9d-d8cb-41143897de33 BootID:915a539b-6597-447b-9fb8-f9355151c9af Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e1:e1:57:09:bd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e1:e1:57:09:bd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:e6:56:bd:c1:5d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 00:59:31.491473 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.491473 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 00:59:31.491585 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.491572 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 00:59:31.495062 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.495038 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 00:59:31.495222 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.495065 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-159.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 00:59:31.495913 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.495902 2582 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 00:59:31.495949 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.495915 2582 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 00:59:31.495949 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.495923 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 00:59:31.495949 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.495938 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 00:59:31.497988 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.497977 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 00:59:31.502757 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.502744 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 14 00:59:31.502913 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.502903 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 00:59:31.502958 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.502936 2582 kubelet.go:486] "Attempting to sync node with API server" Jul 14 00:59:31.502958 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.502953 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 00:59:31.503023 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.502963 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 00:59:31.503023 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.502973 2582 kubelet.go:398] "Adding apiserver pod source" Jul 14 00:59:31.503023 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.502983 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 00:59:31.503436 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.503421 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 00:59:31.503474 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.503459 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 00:59:31.504734 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.504711 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4jqxw" Jul 14 00:59:31.506522 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.506505 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 00:59:31.509600 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.509575 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 00:59:31.509699 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.509624 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 00:59:31.511128 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.511111 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 00:59:31.511183 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.511132 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 00:59:31.511183 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.511139 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 00:59:31.511183 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.511144 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 00:59:31.511183 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.511153 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 00:59:31.511183 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.511159 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 00:59:31.511183 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.511168 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 00:59:31.511183 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.511173 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 00:59:31.511183 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.511180 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 00:59:31.511183 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.511186 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 00:59:31.511442 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.511195 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 00:59:31.511442 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.511201 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 00:59:31.511442 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.511225 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 00:59:31.511442 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.511231 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 00:59:31.511760 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.511740 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4jqxw" Jul 14 00:59:31.513123 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:31.513101 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-159.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 00:59:31.513285 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:31.513265 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 00:59:31.513354 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.513338 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-159.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 00:59:31.514812 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.514798 2582 server.go:1268] "Started kubelet" Jul 14 00:59:31.514931 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.514895 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 00:59:31.514988 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.514920 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 00:59:31.515042 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.514998 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 00:59:31.515743 ip-10-0-143-159 systemd[1]: Started Kubernetes Kubelet. Jul 14 00:59:31.516417 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.516296 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 00:59:31.517731 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.517715 2582 server.go:310] "Adding debug handlers to kubelet server" Jul 14 00:59:31.520472 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:31.520445 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 00:59:31.523378 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.523362 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 00:59:31.523461 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.523433 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 00:59:31.523461 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:31.523445 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 14 00:59:31.524203 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.524186 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 00:59:31.524203 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.524204 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 00:59:31.524354 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.524188 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 00:59:31.524354 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.524294 2582 reconstruct.go:97] "Volume reconstruction finished" Jul 14 00:59:31.524354 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.524302 2582 reconciler.go:29] "Reconciler: start to sync state" Jul 14 00:59:31.524354 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.524366 2582 factory.go:55] Registering systemd factory Jul 14 00:59:31.524692 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.524382 2582 factory.go:223] Registration of the systemd container factory successfully Jul 14 00:59:31.525214 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.525196 2582 factory.go:153] Registering CRI-O factory Jul 14 00:59:31.525214 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.525216 2582 factory.go:223] Registration of the crio container factory successfully Jul 14 00:59:31.525370 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.525274 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 00:59:31.525370 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.525298 2582 factory.go:103] Registering Raw factory Jul 14 00:59:31.525370 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.525314 2582 manager.go:1196] Started watching for new ooms in manager Jul 14 00:59:31.525860 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.525837 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 00:59:31.526040 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.526026 2582 manager.go:349] Starting recovery of all containers Jul 14 00:59:31.528246 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:31.528222 2582 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-159.ec2.internal\" not found" node="ip-10-0-143-159.ec2.internal" Jul 14 00:59:31.537602 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.537586 2582 manager.go:354] Recovery completed Jul 14 00:59:31.542060 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.542047 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 00:59:31.543860 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.543844 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientMemory" Jul 14 00:59:31.543918 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.543874 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 00:59:31.543918 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.543889 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientPID" Jul 14 00:59:31.544433 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.544418 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 00:59:31.544515 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.544433 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 00:59:31.544515 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.544437 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 00:59:31.544515 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.544451 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 14 00:59:31.545187 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.545170 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientMemory" Jul 14 00:59:31.545264 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.545198 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 00:59:31.545264 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.545211 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientPID" Jul 14 00:59:31.546586 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.546575 2582 policy_none.go:49] "None policy: Start" Jul 14 00:59:31.546632 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.546592 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 00:59:31.546632 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.546603 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 00:59:31.548625 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.548614 2582 policy_none.go:47] "Start" Jul 14 00:59:31.589211 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.589008 2582 manager.go:337] "Starting Device Plugin manager" Jul 14 00:59:31.610278 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:31.589251 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 00:59:31.610278 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.589264 2582 server.go:87] "Starting device plugin registration server" Jul 14 00:59:31.610278 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.589683 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 00:59:31.610278 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.589694 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 00:59:31.610278 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.589804 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 00:59:31.610278 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.589894 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 00:59:31.610278 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.589903 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 00:59:31.610278 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:31.591551 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 00:59:31.610278 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:31.591630 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-159.ec2.internal\" not found" Jul 14 00:59:31.685017 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.684928 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 00:59:31.686256 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.686238 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 00:59:31.686304 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.686283 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 00:59:31.686352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.686316 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 00:59:31.686387 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:31.686373 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 00:59:31.688650 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.688628 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 00:59:31.690276 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.690256 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 00:59:31.691043 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.691028 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientMemory" Jul 14 00:59:31.691087 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.691058 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 00:59:31.691087 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.691069 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientPID" Jul 14 00:59:31.691164 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.691101 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-159.ec2.internal" Jul 14 00:59:31.701511 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.701492 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-159.ec2.internal" Jul 14 00:59:31.701614 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:31.701518 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-159.ec2.internal\": node \"ip-10-0-143-159.ec2.internal\" not found" Jul 14 00:59:31.715400 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:31.715371 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 14 00:59:31.786765 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.786710 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal"] Jul 14 00:59:31.786931 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.786811 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 00:59:31.787883 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.787854 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientMemory" Jul 14 00:59:31.788017 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.787898 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 00:59:31.788017 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.787912 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientPID" Jul 14 00:59:31.790268 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.790254 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 00:59:31.790446 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.790430 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 14 00:59:31.790555 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.790469 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 00:59:31.790974 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.790955 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientMemory" Jul 14 00:59:31.791084 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.790960 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientMemory" Jul 14 00:59:31.791084 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.790984 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 00:59:31.791084 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.790999 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientPID" Jul 14 00:59:31.791084 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.791003 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 00:59:31.791084 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.791015 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientPID" Jul 14 00:59:31.793159 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.793144 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" Jul 14 00:59:31.793243 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.793191 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 00:59:31.793906 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.793892 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientMemory" Jul 14 00:59:31.793988 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.793913 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 00:59:31.793988 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.793923 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeHasSufficientPID" Jul 14 00:59:31.808966 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:31.808936 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-159.ec2.internal\" not found" node="ip-10-0-143-159.ec2.internal" Jul 14 00:59:31.812489 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:31.812469 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-159.ec2.internal\" not found" node="ip-10-0-143-159.ec2.internal" Jul 14 00:59:31.815938 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:31.815924 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 14 00:59:31.917052 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:31.917017 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 14 00:59:31.925315 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.925296 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7445fbc0edf80cb2d4f53f5ad08f1d5-config\") pod \"kube-apiserver-proxy-ip-10-0-143-159.ec2.internal\" (UID: \"a7445fbc0edf80cb2d4f53f5ad08f1d5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" Jul 14 00:59:31.925396 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.925336 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/08020b785c0c8b6a8251ff1124de6164-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal\" (UID: \"08020b785c0c8b6a8251ff1124de6164\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 14 00:59:31.925396 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:31.925356 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08020b785c0c8b6a8251ff1124de6164-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal\" (UID: \"08020b785c0c8b6a8251ff1124de6164\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 14 00:59:32.017834 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:32.017762 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 14 00:59:32.026125 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.026099 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7445fbc0edf80cb2d4f53f5ad08f1d5-config\") pod \"kube-apiserver-proxy-ip-10-0-143-159.ec2.internal\" (UID: \"a7445fbc0edf80cb2d4f53f5ad08f1d5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" Jul 14 00:59:32.026171 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.026132 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/08020b785c0c8b6a8251ff1124de6164-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal\" (UID: \"08020b785c0c8b6a8251ff1124de6164\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 14 00:59:32.026203 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.026178 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08020b785c0c8b6a8251ff1124de6164-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal\" (UID: \"08020b785c0c8b6a8251ff1124de6164\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 14 00:59:32.026238 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.026198 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7445fbc0edf80cb2d4f53f5ad08f1d5-config\") pod \"kube-apiserver-proxy-ip-10-0-143-159.ec2.internal\" (UID: \"a7445fbc0edf80cb2d4f53f5ad08f1d5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" Jul 14 00:59:32.026238 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.026211 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08020b785c0c8b6a8251ff1124de6164-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal\" (UID: \"08020b785c0c8b6a8251ff1124de6164\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 14 00:59:32.026238 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.026213 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/08020b785c0c8b6a8251ff1124de6164-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal\" (UID: \"08020b785c0c8b6a8251ff1124de6164\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 14 00:59:32.110317 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.110277 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 14 00:59:32.115405 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.115134 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" Jul 14 00:59:32.118283 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:32.118265 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 14 00:59:32.218561 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:32.218518 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 14 00:59:32.319099 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:32.319018 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-159.ec2.internal\" not found" Jul 14 00:59:32.350884 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.350858 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 00:59:32.423974 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.423947 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" Jul 14 00:59:32.431814 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.431793 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 00:59:32.432972 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.432955 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" Jul 14 00:59:32.443172 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.443150 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 00:59:32.452325 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.452303 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 00:59:32.452477 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.452455 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 00:59:32.452511 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.452485 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 00:59:32.452511 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.452496 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 00:59:32.503652 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.503616 2582 apiserver.go:52] "Watching apiserver" Jul 14 00:59:32.518603 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.518472 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 00:54:31 +0000 UTC" deadline="2028-03-02 11:16:56.373602504 +0000 UTC" Jul 14 00:59:32.518603 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.518523 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14338h17m23.855083697s" Jul 14 00:59:32.520993 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.520969 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 00:59:32.521372 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.521341 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-rmnsp","kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal","openshift-multus/network-metrics-daemon-kch6v","openshift-network-diagnostics/network-check-target-rcg77","openshift-ovn-kubernetes/ovnkube-node-pqg6h","kube-system/konnectivity-agent-pgcsp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz","openshift-cluster-node-tuning-operator/tuned-4pq4q","openshift-dns/node-resolver-5hpkv","openshift-image-registry/node-ca-24x4q","openshift-multus/multus-additional-cni-plugins-7hspr","openshift-multus/multus-hfvn8","openshift-network-operator/iptables-alerter-d4s6j"] Jul 14 00:59:32.523953 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.523937 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 00:59:32.524027 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.523954 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pgcsp" Jul 14 00:59:32.526080 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.526065 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 00:59:32.526178 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.526164 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.526598 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.526581 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 00:59:32.526795 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.526780 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6rr7l\"" Jul 14 00:59:32.528238 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528221 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-52pvx\"" Jul 14 00:59:32.528372 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528254 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 00:59:32.528372 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528284 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-run-k8s-cni-cncf-io\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.528372 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528341 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-hostroot\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.528372 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528358 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d4s6j" Jul 14 00:59:32.528573 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528379 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7d405593-7032-416c-852c-c947b5481d4f-konnectivity-ca\") pod \"konnectivity-agent-pgcsp\" (UID: \"7d405593-7032-416c-852c-c947b5481d4f\") " pod="kube-system/konnectivity-agent-pgcsp" Jul 14 00:59:32.528573 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528403 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-system-cni-dir\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.528573 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528434 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 00:59:32.528573 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528482 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 00:59:32.528744 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528427 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-multus-conf-dir\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.528744 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528605 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-etc-kubernetes\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.528744 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528623 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7d405593-7032-416c-852c-c947b5481d4f-agent-certs\") pod \"konnectivity-agent-pgcsp\" (UID: \"7d405593-7032-416c-852c-c947b5481d4f\") " pod="kube-system/konnectivity-agent-pgcsp" Jul 14 00:59:32.528744 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528640 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-os-release\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.528744 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528655 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-run-netns\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.528744 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528681 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 00:59:32.528744 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528689 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfk79\" (UniqueName: \"kubernetes.io/projected/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-kube-api-access-lfk79\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.528744 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528730 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-cnibin\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.529048 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528758 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-cni-binary-copy\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.529048 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528796 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-multus-socket-dir-parent\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.529048 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528816 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-var-lib-cni-bin\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.529048 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528830 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-var-lib-cni-multus\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.529048 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528852 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-var-lib-kubelet\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.529048 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528873 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-multus-daemon-config\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.529048 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528887 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-run-multus-certs\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.529048 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.528901 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-multus-cni-dir\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.530594 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.530560 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 00:59:32.530689 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.530673 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:32.530773 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:32.530750 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 00:59:32.530848 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.530833 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 00:59:32.530930 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.530914 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zwvl4\"" Jul 14 00:59:32.531101 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.531090 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 00:59:32.532817 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.532800 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:32.532891 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:32.532864 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 00:59:32.533475 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.533460 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 00:59:32.535091 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.535077 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.537298 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.537280 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 00:59:32.537402 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.537282 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.537538 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.537520 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 00:59:32.537607 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.537559 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 00:59:32.537607 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.537603 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7mcvr\"" Jul 14 00:59:32.537705 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.537613 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 00:59:32.537705 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.537637 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 00:59:32.537848 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.537836 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 00:59:32.539624 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.539609 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:32.539680 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:32.539663 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 00:59:32.539868 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.539855 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 00:59:32.540117 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.540102 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9mrb7\"" Jul 14 00:59:32.540346 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.540311 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 00:59:32.541894 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.541878 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.544111 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.544097 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 00:59:32.544240 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.544229 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 00:59:32.544360 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.544349 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tbpcp\"" Jul 14 00:59:32.544483 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.544469 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5hpkv" Jul 14 00:59:32.544576 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.544476 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 00:59:32.546712 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.546689 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kmbpf\"" Jul 14 00:59:32.546995 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.546978 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 00:59:32.547304 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.547284 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-24x4q" Jul 14 00:59:32.548214 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.548196 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 00:59:32.549417 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.549398 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.549485 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.549436 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 00:59:32.549795 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.549781 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 00:59:32.549853 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.549810 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 00:59:32.549987 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.549973 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kzrf6\"" Jul 14 00:59:32.551581 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.551567 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xj68r\"" Jul 14 00:59:32.551755 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.551734 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 00:59:32.551869 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.551853 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 00:59:32.552446 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.552431 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zl9zl" Jul 14 00:59:32.559909 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.559890 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zl9zl" Jul 14 00:59:32.625724 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.625703 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 00:59:32.629581 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629559 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a5c2793-deb1-4de9-953f-28ef081786c7-os-release\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.629691 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629595 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-registration-dir\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.629691 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629620 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-sys\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.629691 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629646 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-run-netns\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.629691 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629668 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-cnibin\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.629901 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629692 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-multus-socket-dir-parent\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.629901 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629701 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-run-netns\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.629901 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629719 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-multus-daemon-config\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.629901 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629747 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-cnibin\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.629901 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629745 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a5c2793-deb1-4de9-953f-28ef081786c7-cnibin\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.629901 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629773 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-multus-socket-dir-parent\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.629901 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629783 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-host\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.629901 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629810 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-run-k8s-cni-cncf-io\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.629901 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629835 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcpwj\" (UniqueName: \"kubernetes.io/projected/09a3f486-7a39-4e01-b913-dbbc3c3c3173-kube-api-access-dcpwj\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.629901 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629859 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-var-lib-openvswitch\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.629901 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629883 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-run-ovn-kubernetes\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.629901 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629888 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-run-k8s-cni-cncf-io\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.629901 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629907 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-sysctl-conf\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629932 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-system-cni-dir\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629957 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-multus-conf-dir\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629982 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a5c2793-deb1-4de9-953f-28ef081786c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.629994 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-system-cni-dir\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630006 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-socket-dir\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630043 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-multus-conf-dir\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630059 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-cni-bin\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630086 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630116 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-os-release\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630142 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfk79\" (UniqueName: \"kubernetes.io/projected/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-kube-api-access-lfk79\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630173 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-cni-netd\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630196 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/24004bce-6659-4fc1-a3e5-8a8d85103612-serviceca\") pod \"node-ca-24x4q\" (UID: \"24004bce-6659-4fc1-a3e5-8a8d85103612\") " pod="openshift-image-registry/node-ca-24x4q" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630217 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-os-release\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630246 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-cni-binary-copy\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630270 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-var-lib-kubelet\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630347 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-var-lib-kubelet\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.630489 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630368 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-run-multus-certs\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630397 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-sys-fs\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630418 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-run-multus-certs\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630431 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c31e51b7-7bee-4ce8-9517-a1e6814a2641-hosts-file\") pod \"node-resolver-5hpkv\" (UID: \"c31e51b7-7bee-4ce8-9517-a1e6814a2641\") " pod="openshift-dns/node-resolver-5hpkv" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630463 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-env-overrides\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630504 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-kubernetes\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630529 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-sysctl-d\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630555 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-multus-cni-dir\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630585 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-hostroot\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630614 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7nq6\" (UniqueName: \"kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6\") pod \"network-check-target-rcg77\" (UID: \"84c558a6-a95a-44c3-8500-3e5d85a08a0a\") " pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630641 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f7953918-10f5-4a00-8b0a-d89348f25314-dbus\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630671 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-multus-cni-dir\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630676 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-hostroot\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630683 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-device-dir\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630706 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c31e51b7-7bee-4ce8-9517-a1e6814a2641-tmp-dir\") pod \"node-resolver-5hpkv\" (UID: \"c31e51b7-7bee-4ce8-9517-a1e6814a2641\") " pod="openshift-dns/node-resolver-5hpkv" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630730 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-kubelet\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630757 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-log-socket\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.631105 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630774 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-etc-kubernetes\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630789 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a5c2793-deb1-4de9-953f-28ef081786c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630804 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8-host-slash\") pod \"iptables-alerter-d4s6j\" (UID: \"6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8\") " pod="openshift-network-operator/iptables-alerter-d4s6j" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630831 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkh8t\" (UniqueName: \"kubernetes.io/projected/6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8-kube-api-access-bkh8t\") pod \"iptables-alerter-d4s6j\" (UID: \"6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8\") " pod="openshift-network-operator/iptables-alerter-d4s6j" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630847 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-ovnkube-config\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630863 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hz9c\" (UniqueName: \"kubernetes.io/projected/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-kube-api-access-9hz9c\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630891 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x8wg\" (UniqueName: \"kubernetes.io/projected/f3495fec-db67-437f-9803-3d2672f63e2e-kube-api-access-2x8wg\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630913 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps7bs\" (UniqueName: \"kubernetes.io/projected/24004bce-6659-4fc1-a3e5-8a8d85103612-kube-api-access-ps7bs\") pod \"node-ca-24x4q\" (UID: \"24004bce-6659-4fc1-a3e5-8a8d85103612\") " pod="openshift-image-registry/node-ca-24x4q" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630930 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-systemd-units\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630944 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-sysconfig\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630961 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f3495fec-db67-437f-9803-3d2672f63e2e-etc-tuned\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.630975 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24004bce-6659-4fc1-a3e5-8a8d85103612-host\") pod \"node-ca-24x4q\" (UID: \"24004bce-6659-4fc1-a3e5-8a8d85103612\") " pod="openshift-image-registry/node-ca-24x4q" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631004 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-var-lib-cni-bin\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631036 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a5c2793-deb1-4de9-953f-28ef081786c7-system-cni-dir\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631076 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-var-lib-cni-bin\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631078 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6a5c2793-deb1-4de9-953f-28ef081786c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.631907 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631115 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f7953918-10f5-4a00-8b0a-d89348f25314-kubelet-config\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631195 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-etc-selinux\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631272 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-etc-kubernetes\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631271 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-systemd\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631302 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-lib-modules\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631346 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6a5c2793-deb1-4de9-953f-28ef081786c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631374 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bcm9\" (UniqueName: \"kubernetes.io/projected/6a5c2793-deb1-4de9-953f-28ef081786c7-kube-api-access-9bcm9\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631409 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfhss\" (UniqueName: \"kubernetes.io/projected/3c724b42-2885-4687-970d-9ef08ddf10ff-kube-api-access-xfhss\") pod \"network-metrics-daemon-kch6v\" (UID: \"3c724b42-2885-4687-970d-9ef08ddf10ff\") " pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631435 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631459 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631490 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-slash\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631514 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-ovn-node-metrics-cert\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631537 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-ovnkube-script-lib\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631569 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7d405593-7032-416c-852c-c947b5481d4f-konnectivity-ca\") pod \"konnectivity-agent-pgcsp\" (UID: \"7d405593-7032-416c-852c-c947b5481d4f\") " pod="kube-system/konnectivity-agent-pgcsp" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631595 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjf9t\" (UniqueName: \"kubernetes.io/projected/c31e51b7-7bee-4ce8-9517-a1e6814a2641-kube-api-access-cjf9t\") pod \"node-resolver-5hpkv\" (UID: \"c31e51b7-7bee-4ce8-9517-a1e6814a2641\") " pod="openshift-dns/node-resolver-5hpkv" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631617 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-run-netns\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.632418 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631640 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-etc-openvswitch\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.632857 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631665 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7d405593-7032-416c-852c-c947b5481d4f-agent-certs\") pod \"konnectivity-agent-pgcsp\" (UID: \"7d405593-7032-416c-852c-c947b5481d4f\") " pod="kube-system/konnectivity-agent-pgcsp" Jul 14 00:59:32.632857 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631689 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8-iptables-alerter-script\") pod \"iptables-alerter-d4s6j\" (UID: \"6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8\") " pod="openshift-network-operator/iptables-alerter-d4s6j" Jul 14 00:59:32.632857 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631728 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-node-log\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.632857 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631749 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-var-lib-kubelet\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.632857 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631778 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3495fec-db67-437f-9803-3d2672f63e2e-tmp\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.632857 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631801 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-var-lib-cni-multus\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.632857 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631816 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs\") pod \"network-metrics-daemon-kch6v\" (UID: \"3c724b42-2885-4687-970d-9ef08ddf10ff\") " pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:32.632857 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631830 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-run-systemd\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.632857 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631841 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-host-var-lib-cni-multus\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.632857 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631849 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-run-openvswitch\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.632857 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631877 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-run-ovn\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.632857 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631893 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-modprobe-d\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.632857 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.631908 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-run\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.632857 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.632770 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 00:59:32.633270 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.632886 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7d405593-7032-416c-852c-c947b5481d4f-konnectivity-ca\") pod \"konnectivity-agent-pgcsp\" (UID: \"7d405593-7032-416c-852c-c947b5481d4f\") " pod="kube-system/konnectivity-agent-pgcsp" Jul 14 00:59:32.633270 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.633063 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-cni-binary-copy\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.633509 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.633493 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-multus-daemon-config\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.636242 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.636227 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7d405593-7032-416c-852c-c947b5481d4f-agent-certs\") pod \"konnectivity-agent-pgcsp\" (UID: \"7d405593-7032-416c-852c-c947b5481d4f\") " pod="kube-system/konnectivity-agent-pgcsp" Jul 14 00:59:32.637246 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.637230 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfk79\" (UniqueName: \"kubernetes.io/projected/7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8-kube-api-access-lfk79\") pod \"multus-hfvn8\" (UID: \"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8\") " pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.677577 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.677552 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 00:59:32.732582 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732548 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjf9t\" (UniqueName: \"kubernetes.io/projected/c31e51b7-7bee-4ce8-9517-a1e6814a2641-kube-api-access-cjf9t\") pod \"node-resolver-5hpkv\" (UID: \"c31e51b7-7bee-4ce8-9517-a1e6814a2641\") " pod="openshift-dns/node-resolver-5hpkv" Jul 14 00:59:32.732582 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732579 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-run-netns\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.732820 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732599 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-etc-openvswitch\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.732820 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732624 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8-iptables-alerter-script\") pod \"iptables-alerter-d4s6j\" (UID: \"6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8\") " pod="openshift-network-operator/iptables-alerter-d4s6j" Jul 14 00:59:32.732820 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732649 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-node-log\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.732820 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732671 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-var-lib-kubelet\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.732820 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732681 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-run-netns\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.732820 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732692 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3495fec-db67-437f-9803-3d2672f63e2e-tmp\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.732820 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732745 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs\") pod \"network-metrics-daemon-kch6v\" (UID: \"3c724b42-2885-4687-970d-9ef08ddf10ff\") " pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:32.732820 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732771 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-run-systemd\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.732820 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732795 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-run-openvswitch\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.732820 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732812 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-run-ovn\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.732820 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732827 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-modprobe-d\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732841 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-run\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732860 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a5c2793-deb1-4de9-953f-28ef081786c7-os-release\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732882 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-registration-dir\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732903 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-sys\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732914 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-etc-openvswitch\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732934 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a5c2793-deb1-4de9-953f-28ef081786c7-cnibin\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732958 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-host\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732983 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcpwj\" (UniqueName: \"kubernetes.io/projected/09a3f486-7a39-4e01-b913-dbbc3c3c3173-kube-api-access-dcpwj\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733006 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-modprobe-d\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733059 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-run\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733054 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-var-lib-openvswitch\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733109 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-var-lib-kubelet\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733118 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a5c2793-deb1-4de9-953f-28ef081786c7-os-release\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733149 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-registration-dir\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:32.733162 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733173 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-run-openvswitch\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733197 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-sys\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.733395 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733221 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-run-systemd\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:32.733240 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs podName:3c724b42-2885-4687-970d-9ef08ddf10ff nodeName:}" failed. No retries permitted until 2026-07-14 00:59:33.233205268 +0000 UTC m=+2.136060750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs") pod "network-metrics-daemon-kch6v" (UID: "3c724b42-2885-4687-970d-9ef08ddf10ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733268 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-host\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733226 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a5c2793-deb1-4de9-953f-28ef081786c7-cnibin\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.732884 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-node-log\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733009 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-var-lib-openvswitch\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733297 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8-iptables-alerter-script\") pod \"iptables-alerter-d4s6j\" (UID: \"6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8\") " pod="openshift-network-operator/iptables-alerter-d4s6j" Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733365 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-run-ovn-kubernetes\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733340 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-run-ovn\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733444 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-run-ovn-kubernetes\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733394 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-sysctl-conf\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733482 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-sysctl-conf\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733510 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a5c2793-deb1-4de9-953f-28ef081786c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733534 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-socket-dir\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733557 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-cni-bin\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733582 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.734352 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733609 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-cni-netd\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733619 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-socket-dir\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733632 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/24004bce-6659-4fc1-a3e5-8a8d85103612-serviceca\") pod \"node-ca-24x4q\" (UID: \"24004bce-6659-4fc1-a3e5-8a8d85103612\") " pod="openshift-image-registry/node-ca-24x4q" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733640 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-cni-bin\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733662 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-sys-fs\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733677 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733690 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c31e51b7-7bee-4ce8-9517-a1e6814a2641-hosts-file\") pod \"node-resolver-5hpkv\" (UID: \"c31e51b7-7bee-4ce8-9517-a1e6814a2641\") " pod="openshift-dns/node-resolver-5hpkv" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733696 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-cni-netd\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733756 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-sys-fs\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733884 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c31e51b7-7bee-4ce8-9517-a1e6814a2641-hosts-file\") pod \"node-resolver-5hpkv\" (UID: \"c31e51b7-7bee-4ce8-9517-a1e6814a2641\") " pod="openshift-dns/node-resolver-5hpkv" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733884 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-env-overrides\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733928 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-kubernetes\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733953 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-sysctl-d\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.733980 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7nq6\" (UniqueName: \"kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6\") pod \"network-check-target-rcg77\" (UID: \"84c558a6-a95a-44c3-8500-3e5d85a08a0a\") " pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734042 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f7953918-10f5-4a00-8b0a-d89348f25314-dbus\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734067 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-device-dir\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734065 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-kubernetes\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.735120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734092 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c31e51b7-7bee-4ce8-9517-a1e6814a2641-tmp-dir\") pod \"node-resolver-5hpkv\" (UID: \"c31e51b7-7bee-4ce8-9517-a1e6814a2641\") " pod="openshift-dns/node-resolver-5hpkv" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734119 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-kubelet\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734134 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a5c2793-deb1-4de9-953f-28ef081786c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734142 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-log-socket\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734177 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-log-socket\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734177 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/24004bce-6659-4fc1-a3e5-8a8d85103612-serviceca\") pod \"node-ca-24x4q\" (UID: \"24004bce-6659-4fc1-a3e5-8a8d85103612\") " pod="openshift-image-registry/node-ca-24x4q" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734198 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a5c2793-deb1-4de9-953f-28ef081786c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734230 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f7953918-10f5-4a00-8b0a-d89348f25314-dbus\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734267 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8-host-slash\") pod \"iptables-alerter-d4s6j\" (UID: \"6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8\") " pod="openshift-network-operator/iptables-alerter-d4s6j" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734353 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-kubelet\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734351 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-env-overrides\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734230 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8-host-slash\") pod \"iptables-alerter-d4s6j\" (UID: \"6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8\") " pod="openshift-network-operator/iptables-alerter-d4s6j" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734398 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-device-dir\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734407 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkh8t\" (UniqueName: \"kubernetes.io/projected/6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8-kube-api-access-bkh8t\") pod \"iptables-alerter-d4s6j\" (UID: \"6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8\") " pod="openshift-network-operator/iptables-alerter-d4s6j" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734428 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-sysctl-d\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734432 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-ovnkube-config\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734475 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hz9c\" (UniqueName: \"kubernetes.io/projected/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-kube-api-access-9hz9c\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.735824 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734509 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c31e51b7-7bee-4ce8-9517-a1e6814a2641-tmp-dir\") pod \"node-resolver-5hpkv\" (UID: \"c31e51b7-7bee-4ce8-9517-a1e6814a2641\") " pod="openshift-dns/node-resolver-5hpkv" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734529 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x8wg\" (UniqueName: \"kubernetes.io/projected/f3495fec-db67-437f-9803-3d2672f63e2e-kube-api-access-2x8wg\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734553 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a5c2793-deb1-4de9-953f-28ef081786c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734574 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps7bs\" (UniqueName: \"kubernetes.io/projected/24004bce-6659-4fc1-a3e5-8a8d85103612-kube-api-access-ps7bs\") pod \"node-ca-24x4q\" (UID: \"24004bce-6659-4fc1-a3e5-8a8d85103612\") " pod="openshift-image-registry/node-ca-24x4q" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734726 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-systemd-units\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734753 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-sysconfig\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734778 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f3495fec-db67-437f-9803-3d2672f63e2e-etc-tuned\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734783 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-systemd-units\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734801 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24004bce-6659-4fc1-a3e5-8a8d85103612-host\") pod \"node-ca-24x4q\" (UID: \"24004bce-6659-4fc1-a3e5-8a8d85103612\") " pod="openshift-image-registry/node-ca-24x4q" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734835 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24004bce-6659-4fc1-a3e5-8a8d85103612-host\") pod \"node-ca-24x4q\" (UID: \"24004bce-6659-4fc1-a3e5-8a8d85103612\") " pod="openshift-image-registry/node-ca-24x4q" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734839 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a5c2793-deb1-4de9-953f-28ef081786c7-system-cni-dir\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734851 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-sysconfig\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734869 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6a5c2793-deb1-4de9-953f-28ef081786c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734877 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a5c2793-deb1-4de9-953f-28ef081786c7-system-cni-dir\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734907 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f7953918-10f5-4a00-8b0a-d89348f25314-kubelet-config\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734932 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-etc-selinux\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734944 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-ovnkube-config\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.736392 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734954 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-systemd\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734975 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f7953918-10f5-4a00-8b0a-d89348f25314-kubelet-config\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.734980 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-lib-modules\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735019 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6a5c2793-deb1-4de9-953f-28ef081786c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735044 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-etc-selinux\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735049 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bcm9\" (UniqueName: \"kubernetes.io/projected/6a5c2793-deb1-4de9-953f-28ef081786c7-kube-api-access-9bcm9\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735067 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-lib-modules\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735075 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfhss\" (UniqueName: \"kubernetes.io/projected/3c724b42-2885-4687-970d-9ef08ddf10ff-kube-api-access-xfhss\") pod \"network-metrics-daemon-kch6v\" (UID: \"3c724b42-2885-4687-970d-9ef08ddf10ff\") " pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735100 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f3495fec-db67-437f-9803-3d2672f63e2e-etc-systemd\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735217 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735272 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:32.735287 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735313 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-slash\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:32.735364 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret podName:f7953918-10f5-4a00-8b0a-d89348f25314 nodeName:}" failed. No retries permitted until 2026-07-14 00:59:33.235344257 +0000 UTC m=+2.138199734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret") pod "global-pull-secret-syncer-rmnsp" (UID: "f7953918-10f5-4a00-8b0a-d89348f25314") : object "kube-system"/"original-pull-secret" not registered Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735373 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3495fec-db67-437f-9803-3d2672f63e2e-tmp\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735385 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-ovn-node-metrics-cert\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735392 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-host-slash\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.736855 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735417 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09a3f486-7a39-4e01-b913-dbbc3c3c3173-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.737488 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735417 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-ovnkube-script-lib\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.737488 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735524 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6a5c2793-deb1-4de9-953f-28ef081786c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.737488 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735544 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6a5c2793-deb1-4de9-953f-28ef081786c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.737488 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.735961 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-ovnkube-script-lib\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.737684 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.737551 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f3495fec-db67-437f-9803-3d2672f63e2e-etc-tuned\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.737684 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.737615 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-ovn-node-metrics-cert\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.741129 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:32.740729 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 00:59:32.741129 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:32.740817 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 00:59:32.741129 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:32.740831 2582 projected.go:196] Error preparing data for projected volume kube-api-access-m7nq6 for pod openshift-network-diagnostics/network-check-target-rcg77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:59:32.741129 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:32.740881 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6 podName:84c558a6-a95a-44c3-8500-3e5d85a08a0a nodeName:}" failed. No retries permitted until 2026-07-14 00:59:33.240865733 +0000 UTC m=+2.143721212 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m7nq6" (UniqueName: "kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6") pod "network-check-target-rcg77" (UID: "84c558a6-a95a-44c3-8500-3e5d85a08a0a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:59:32.741742 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.741719 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjf9t\" (UniqueName: \"kubernetes.io/projected/c31e51b7-7bee-4ce8-9517-a1e6814a2641-kube-api-access-cjf9t\") pod \"node-resolver-5hpkv\" (UID: \"c31e51b7-7bee-4ce8-9517-a1e6814a2641\") " pod="openshift-dns/node-resolver-5hpkv" Jul 14 00:59:32.742868 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.742845 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcpwj\" (UniqueName: \"kubernetes.io/projected/09a3f486-7a39-4e01-b913-dbbc3c3c3173-kube-api-access-dcpwj\") pod \"aws-ebs-csi-driver-node-bvswz\" (UID: \"09a3f486-7a39-4e01-b913-dbbc3c3c3173\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.743353 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.743315 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x8wg\" (UniqueName: \"kubernetes.io/projected/f3495fec-db67-437f-9803-3d2672f63e2e-kube-api-access-2x8wg\") pod \"tuned-4pq4q\" (UID: \"f3495fec-db67-437f-9803-3d2672f63e2e\") " pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.743577 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.743557 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfhss\" (UniqueName: \"kubernetes.io/projected/3c724b42-2885-4687-970d-9ef08ddf10ff-kube-api-access-xfhss\") pod \"network-metrics-daemon-kch6v\" (UID: \"3c724b42-2885-4687-970d-9ef08ddf10ff\") " pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:32.743659 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.743596 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkh8t\" (UniqueName: \"kubernetes.io/projected/6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8-kube-api-access-bkh8t\") pod \"iptables-alerter-d4s6j\" (UID: \"6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8\") " pod="openshift-network-operator/iptables-alerter-d4s6j" Jul 14 00:59:32.743710 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.743696 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps7bs\" (UniqueName: \"kubernetes.io/projected/24004bce-6659-4fc1-a3e5-8a8d85103612-kube-api-access-ps7bs\") pod \"node-ca-24x4q\" (UID: \"24004bce-6659-4fc1-a3e5-8a8d85103612\") " pod="openshift-image-registry/node-ca-24x4q" Jul 14 00:59:32.744078 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.744062 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bcm9\" (UniqueName: \"kubernetes.io/projected/6a5c2793-deb1-4de9-953f-28ef081786c7-kube-api-access-9bcm9\") pod \"multus-additional-cni-plugins-7hspr\" (UID: \"6a5c2793-deb1-4de9-953f-28ef081786c7\") " pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.744383 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.744367 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hz9c\" (UniqueName: \"kubernetes.io/projected/3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f-kube-api-access-9hz9c\") pod \"ovnkube-node-pqg6h\" (UID: \"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.856759 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.856685 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pgcsp" Jul 14 00:59:32.878198 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.878169 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hfvn8" Jul 14 00:59:32.900200 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:32.900148 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bd7b9b5_3fdf_47de_8717_bb5c6f150fb8.slice/crio-b8c5704e2eb5b58062b1edf9767ef3f213e01d155b8aa56366c7061491588b50 WatchSource:0}: Error finding container b8c5704e2eb5b58062b1edf9767ef3f213e01d155b8aa56366c7061491588b50: Status 404 returned error can't find the container with id b8c5704e2eb5b58062b1edf9767ef3f213e01d155b8aa56366c7061491588b50 Jul 14 00:59:32.900808 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:32.900780 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7445fbc0edf80cb2d4f53f5ad08f1d5.slice/crio-b0efa97427edf65a3b23c9517486dc8306dd3b1b5ae5289a6761578d8bdb4bd7 WatchSource:0}: Error finding container b0efa97427edf65a3b23c9517486dc8306dd3b1b5ae5289a6761578d8bdb4bd7: Status 404 returned error can't find the container with id b0efa97427edf65a3b23c9517486dc8306dd3b1b5ae5289a6761578d8bdb4bd7 Jul 14 00:59:32.901498 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:32.901469 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08020b785c0c8b6a8251ff1124de6164.slice/crio-f7eb454d3861f8d98a49d80a939b3f0f91b04566483ffa3f3466e544fbaf3aab WatchSource:0}: Error finding container f7eb454d3861f8d98a49d80a939b3f0f91b04566483ffa3f3466e544fbaf3aab: Status 404 returned error can't find the container with id f7eb454d3861f8d98a49d80a939b3f0f91b04566483ffa3f3466e544fbaf3aab Jul 14 00:59:32.901565 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.901548 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d4s6j" Jul 14 00:59:32.905378 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.905362 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 00:59:32.911295 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:32.911265 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc9c8a4_9f6d_481b_92e5_ae3e76527aa8.slice/crio-ef5bee522363bbeae3ee86cf596f14abd08394fd6f4bd12c6fe96bb75c1dd215 WatchSource:0}: Error finding container ef5bee522363bbeae3ee86cf596f14abd08394fd6f4bd12c6fe96bb75c1dd215: Status 404 returned error can't find the container with id ef5bee522363bbeae3ee86cf596f14abd08394fd6f4bd12c6fe96bb75c1dd215 Jul 14 00:59:32.925925 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.925903 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:32.934567 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:32.934544 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b930a04_b1ae_47ac_ab05_d8acf6d4eb7f.slice/crio-8f0a4c055a3f6c2fbb06c97799fd78cd0b2d7d49fc29988182267742b83192df WatchSource:0}: Error finding container 8f0a4c055a3f6c2fbb06c97799fd78cd0b2d7d49fc29988182267742b83192df: Status 404 returned error can't find the container with id 8f0a4c055a3f6c2fbb06c97799fd78cd0b2d7d49fc29988182267742b83192df Jul 14 00:59:32.945210 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.945191 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" Jul 14 00:59:32.953887 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:32.953864 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3495fec_db67_437f_9803_3d2672f63e2e.slice/crio-befd4897974b5236ce58800afe561cd98b66e8a682aab8c4a08fd9c1956d0e4a WatchSource:0}: Error finding container befd4897974b5236ce58800afe561cd98b66e8a682aab8c4a08fd9c1956d0e4a: Status 404 returned error can't find the container with id befd4897974b5236ce58800afe561cd98b66e8a682aab8c4a08fd9c1956d0e4a Jul 14 00:59:32.955225 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.955208 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" Jul 14 00:59:32.963938 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:32.963914 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09a3f486_7a39_4e01_b913_dbbc3c3c3173.slice/crio-e9030eff9dc45fbdc3c6f390cd269d73eab8a4554174cb86c6ea60ea077841ea WatchSource:0}: Error finding container e9030eff9dc45fbdc3c6f390cd269d73eab8a4554174cb86c6ea60ea077841ea: Status 404 returned error can't find the container with id e9030eff9dc45fbdc3c6f390cd269d73eab8a4554174cb86c6ea60ea077841ea Jul 14 00:59:32.974869 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.974850 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5hpkv" Jul 14 00:59:32.980832 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.980814 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-24x4q" Jul 14 00:59:32.982945 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:32.982924 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc31e51b7_7bee_4ce8_9517_a1e6814a2641.slice/crio-56317a2c829b756ed67280cc7d6d10c0ada134f08fd879a898e4fc644f7fdf49 WatchSource:0}: Error finding container 56317a2c829b756ed67280cc7d6d10c0ada134f08fd879a898e4fc644f7fdf49: Status 404 returned error can't find the container with id 56317a2c829b756ed67280cc7d6d10c0ada134f08fd879a898e4fc644f7fdf49 Jul 14 00:59:32.983589 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:32.983576 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7hspr" Jul 14 00:59:32.990187 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:32.990166 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24004bce_6659_4fc1_a3e5_8a8d85103612.slice/crio-f461dac139b251d95af5a71a9af8bf86c4593d0efd7cf9275330e3db0e8230a7 WatchSource:0}: Error finding container f461dac139b251d95af5a71a9af8bf86c4593d0efd7cf9275330e3db0e8230a7: Status 404 returned error can't find the container with id f461dac139b251d95af5a71a9af8bf86c4593d0efd7cf9275330e3db0e8230a7 Jul 14 00:59:32.992928 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:32.992903 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a5c2793_deb1_4de9_953f_28ef081786c7.slice/crio-ca004b0cbebba477646d6d6956e933c15969e5947c240c6ceb2bb16b11ea3ae8 WatchSource:0}: Error finding container ca004b0cbebba477646d6d6956e933c15969e5947c240c6ceb2bb16b11ea3ae8: Status 404 returned error can't find the container with id ca004b0cbebba477646d6d6956e933c15969e5947c240c6ceb2bb16b11ea3ae8 Jul 14 00:59:33.239144 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.239058 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:33.239144 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.239095 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs\") pod \"network-metrics-daemon-kch6v\" (UID: \"3c724b42-2885-4687-970d-9ef08ddf10ff\") " pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:33.239313 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:33.239207 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:59:33.239313 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:33.239209 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 00:59:33.239313 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:33.239273 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret podName:f7953918-10f5-4a00-8b0a-d89348f25314 nodeName:}" failed. No retries permitted until 2026-07-14 00:59:34.239258275 +0000 UTC m=+3.142113753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret") pod "global-pull-secret-syncer-rmnsp" (UID: "f7953918-10f5-4a00-8b0a-d89348f25314") : object "kube-system"/"original-pull-secret" not registered Jul 14 00:59:33.239313 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:33.239287 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs podName:3c724b42-2885-4687-970d-9ef08ddf10ff nodeName:}" failed. No retries permitted until 2026-07-14 00:59:34.239280875 +0000 UTC m=+3.142136352 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs") pod "network-metrics-daemon-kch6v" (UID: "3c724b42-2885-4687-970d-9ef08ddf10ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:59:33.329652 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.329615 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 00:59:33.337878 ip-10-0-143-159 kubenswrapper[2582]: W0714 00:59:33.337840 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d405593_7032_416c_852c_c947b5481d4f.slice/crio-ea6fa31855d3ccbe88f4611f8fe79114c2d45b7b0f07a14716d1e13c985f8ba2 WatchSource:0}: Error finding container ea6fa31855d3ccbe88f4611f8fe79114c2d45b7b0f07a14716d1e13c985f8ba2: Status 404 returned error can't find the container with id ea6fa31855d3ccbe88f4611f8fe79114c2d45b7b0f07a14716d1e13c985f8ba2 Jul 14 00:59:33.339435 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.339399 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7nq6\" (UniqueName: \"kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6\") pod \"network-check-target-rcg77\" (UID: \"84c558a6-a95a-44c3-8500-3e5d85a08a0a\") " pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:33.339554 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:33.339522 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 00:59:33.339554 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:33.339537 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 00:59:33.339554 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:33.339546 2582 projected.go:196] Error preparing data for projected volume kube-api-access-m7nq6 for pod openshift-network-diagnostics/network-check-target-rcg77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:59:33.339771 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:33.339599 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6 podName:84c558a6-a95a-44c3-8500-3e5d85a08a0a nodeName:}" failed. No retries permitted until 2026-07-14 00:59:34.339581737 +0000 UTC m=+3.242437231 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7nq6" (UniqueName: "kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6") pod "network-check-target-rcg77" (UID: "84c558a6-a95a-44c3-8500-3e5d85a08a0a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:59:33.563286 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.563175 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 00:54:32 +0000 UTC" deadline="2027-12-16 08:08:49.880238773 +0000 UTC" Jul 14 00:59:33.563286 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.563225 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12487h9m16.317016287s" Jul 14 00:59:33.693768 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.693740 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:33.693910 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:33.693871 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 00:59:33.694710 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.694545 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:33.694710 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:33.694641 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 00:59:33.703988 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.703932 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-24x4q" event={"ID":"24004bce-6659-4fc1-a3e5-8a8d85103612","Type":"ContainerStarted","Data":"f461dac139b251d95af5a71a9af8bf86c4593d0efd7cf9275330e3db0e8230a7"} Jul 14 00:59:33.719438 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.719399 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" event={"ID":"f3495fec-db67-437f-9803-3d2672f63e2e","Type":"ContainerStarted","Data":"befd4897974b5236ce58800afe561cd98b66e8a682aab8c4a08fd9c1956d0e4a"} Jul 14 00:59:33.730858 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.730819 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" event={"ID":"08020b785c0c8b6a8251ff1124de6164","Type":"ContainerStarted","Data":"f7eb454d3861f8d98a49d80a939b3f0f91b04566483ffa3f3466e544fbaf3aab"} Jul 14 00:59:33.746382 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.746341 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hfvn8" event={"ID":"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8","Type":"ContainerStarted","Data":"b8c5704e2eb5b58062b1edf9767ef3f213e01d155b8aa56366c7061491588b50"} Jul 14 00:59:33.748995 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.748963 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pgcsp" event={"ID":"7d405593-7032-416c-852c-c947b5481d4f","Type":"ContainerStarted","Data":"ea6fa31855d3ccbe88f4611f8fe79114c2d45b7b0f07a14716d1e13c985f8ba2"} Jul 14 00:59:33.769266 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.769226 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5hpkv" event={"ID":"c31e51b7-7bee-4ce8-9517-a1e6814a2641","Type":"ContainerStarted","Data":"56317a2c829b756ed67280cc7d6d10c0ada134f08fd879a898e4fc644f7fdf49"} Jul 14 00:59:33.774661 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.774547 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" event={"ID":"09a3f486-7a39-4e01-b913-dbbc3c3c3173","Type":"ContainerStarted","Data":"e9030eff9dc45fbdc3c6f390cd269d73eab8a4554174cb86c6ea60ea077841ea"} Jul 14 00:59:33.788192 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.788030 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" event={"ID":"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f","Type":"ContainerStarted","Data":"8f0a4c055a3f6c2fbb06c97799fd78cd0b2d7d49fc29988182267742b83192df"} Jul 14 00:59:33.820607 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.820516 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d4s6j" event={"ID":"6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8","Type":"ContainerStarted","Data":"ef5bee522363bbeae3ee86cf596f14abd08394fd6f4bd12c6fe96bb75c1dd215"} Jul 14 00:59:33.837922 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.837878 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" event={"ID":"a7445fbc0edf80cb2d4f53f5ad08f1d5","Type":"ContainerStarted","Data":"b0efa97427edf65a3b23c9517486dc8306dd3b1b5ae5289a6761578d8bdb4bd7"} Jul 14 00:59:33.841836 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.841797 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7hspr" event={"ID":"6a5c2793-deb1-4de9-953f-28ef081786c7","Type":"ContainerStarted","Data":"ca004b0cbebba477646d6d6956e933c15969e5947c240c6ceb2bb16b11ea3ae8"} Jul 14 00:59:33.964761 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:33.964041 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 00:59:34.248829 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:34.248734 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:34.248829 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:34.248803 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs\") pod \"network-metrics-daemon-kch6v\" (UID: \"3c724b42-2885-4687-970d-9ef08ddf10ff\") " pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:34.249065 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:34.248904 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 00:59:34.249065 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:34.248953 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:59:34.249065 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:34.248976 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret podName:f7953918-10f5-4a00-8b0a-d89348f25314 nodeName:}" failed. No retries permitted until 2026-07-14 00:59:36.248960139 +0000 UTC m=+5.151815630 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret") pod "global-pull-secret-syncer-rmnsp" (UID: "f7953918-10f5-4a00-8b0a-d89348f25314") : object "kube-system"/"original-pull-secret" not registered Jul 14 00:59:34.249065 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:34.249012 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs podName:3c724b42-2885-4687-970d-9ef08ddf10ff nodeName:}" failed. No retries permitted until 2026-07-14 00:59:36.24899534 +0000 UTC m=+5.151850821 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs") pod "network-metrics-daemon-kch6v" (UID: "3c724b42-2885-4687-970d-9ef08ddf10ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:59:34.349955 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:34.349916 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7nq6\" (UniqueName: \"kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6\") pod \"network-check-target-rcg77\" (UID: \"84c558a6-a95a-44c3-8500-3e5d85a08a0a\") " pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:34.350150 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:34.350113 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 00:59:34.350150 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:34.350133 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 00:59:34.350150 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:34.350146 2582 projected.go:196] Error preparing data for projected volume kube-api-access-m7nq6 for pod openshift-network-diagnostics/network-check-target-rcg77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:59:34.350304 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:34.350208 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6 podName:84c558a6-a95a-44c3-8500-3e5d85a08a0a nodeName:}" failed. No retries permitted until 2026-07-14 00:59:36.350188308 +0000 UTC m=+5.253043810 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7nq6" (UniqueName: "kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6") pod "network-check-target-rcg77" (UID: "84c558a6-a95a-44c3-8500-3e5d85a08a0a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:59:34.564666 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:34.563584 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 00:54:32 +0000 UTC" deadline="2028-03-19 17:58:42.316177662 +0000 UTC" Jul 14 00:59:34.564666 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:34.563624 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14752h59m7.752558375s" Jul 14 00:59:34.687079 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:34.686905 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:34.687079 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:34.687049 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 00:59:34.835843 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:34.835770 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 00:59:35.689889 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:35.689860 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:35.690396 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:35.689990 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 00:59:35.690396 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:35.690050 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:35.690396 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:35.690122 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 00:59:36.268808 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:36.268767 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:36.269003 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:36.268829 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs\") pod \"network-metrics-daemon-kch6v\" (UID: \"3c724b42-2885-4687-970d-9ef08ddf10ff\") " pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:36.269079 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:36.269034 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:59:36.269133 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:36.269116 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs podName:3c724b42-2885-4687-970d-9ef08ddf10ff nodeName:}" failed. No retries permitted until 2026-07-14 00:59:40.269095283 +0000 UTC m=+9.171950763 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs") pod "network-metrics-daemon-kch6v" (UID: "3c724b42-2885-4687-970d-9ef08ddf10ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:59:36.269551 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:36.269529 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 00:59:36.269654 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:36.269584 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret podName:f7953918-10f5-4a00-8b0a-d89348f25314 nodeName:}" failed. No retries permitted until 2026-07-14 00:59:40.269568997 +0000 UTC m=+9.172424496 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret") pod "global-pull-secret-syncer-rmnsp" (UID: "f7953918-10f5-4a00-8b0a-d89348f25314") : object "kube-system"/"original-pull-secret" not registered Jul 14 00:59:36.370347 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:36.370284 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7nq6\" (UniqueName: \"kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6\") pod \"network-check-target-rcg77\" (UID: \"84c558a6-a95a-44c3-8500-3e5d85a08a0a\") " pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:36.370549 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:36.370534 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 00:59:36.370612 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:36.370556 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 00:59:36.370612 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:36.370569 2582 projected.go:196] Error preparing data for projected volume kube-api-access-m7nq6 for pod openshift-network-diagnostics/network-check-target-rcg77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:59:36.370708 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:36.370627 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6 podName:84c558a6-a95a-44c3-8500-3e5d85a08a0a nodeName:}" failed. No retries permitted until 2026-07-14 00:59:40.370607442 +0000 UTC m=+9.273462920 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7nq6" (UniqueName: "kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6") pod "network-check-target-rcg77" (UID: "84c558a6-a95a-44c3-8500-3e5d85a08a0a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:59:36.687246 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:36.687163 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:36.687443 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:36.687333 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 00:59:37.687871 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:37.687432 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:37.687871 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:37.687580 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 00:59:37.697016 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:37.696919 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:37.697418 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:37.697385 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 00:59:38.686583 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:38.686543 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:38.686760 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:38.686694 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 00:59:39.690256 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:39.690223 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:39.690674 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:39.690373 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 00:59:39.690674 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:39.690425 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:39.690674 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:39.690534 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 00:59:40.304433 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:40.304381 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:40.304623 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:40.304441 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs\") pod \"network-metrics-daemon-kch6v\" (UID: \"3c724b42-2885-4687-970d-9ef08ddf10ff\") " pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:40.304623 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:40.304593 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:59:40.304734 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:40.304656 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs podName:3c724b42-2885-4687-970d-9ef08ddf10ff nodeName:}" failed. No retries permitted until 2026-07-14 00:59:48.304636368 +0000 UTC m=+17.207491845 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs") pod "network-metrics-daemon-kch6v" (UID: "3c724b42-2885-4687-970d-9ef08ddf10ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:59:40.305128 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:40.305041 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 00:59:40.305128 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:40.305094 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret podName:f7953918-10f5-4a00-8b0a-d89348f25314 nodeName:}" failed. No retries permitted until 2026-07-14 00:59:48.305079465 +0000 UTC m=+17.207934960 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret") pod "global-pull-secret-syncer-rmnsp" (UID: "f7953918-10f5-4a00-8b0a-d89348f25314") : object "kube-system"/"original-pull-secret" not registered Jul 14 00:59:40.405712 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:40.405618 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7nq6\" (UniqueName: \"kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6\") pod \"network-check-target-rcg77\" (UID: \"84c558a6-a95a-44c3-8500-3e5d85a08a0a\") " pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:40.405873 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:40.405790 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 00:59:40.405873 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:40.405819 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 00:59:40.405873 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:40.405832 2582 projected.go:196] Error preparing data for projected volume kube-api-access-m7nq6 for pod openshift-network-diagnostics/network-check-target-rcg77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:59:40.406036 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:40.405894 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6 podName:84c558a6-a95a-44c3-8500-3e5d85a08a0a nodeName:}" failed. No retries permitted until 2026-07-14 00:59:48.405872973 +0000 UTC m=+17.308728450 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7nq6" (UniqueName: "kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6") pod "network-check-target-rcg77" (UID: "84c558a6-a95a-44c3-8500-3e5d85a08a0a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:59:40.687286 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:40.687190 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:40.687450 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:40.687382 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 00:59:41.686929 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:41.686892 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:41.687387 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:41.687033 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 00:59:41.688501 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:41.688477 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:41.688613 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:41.688584 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 00:59:42.687543 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:42.687502 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:42.687935 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:42.687669 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 00:59:43.687364 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:43.687312 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:43.687523 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:43.687438 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 00:59:43.687523 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:43.687506 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:43.687864 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:43.687629 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 00:59:44.686558 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:44.686522 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:44.686714 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:44.686685 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 00:59:45.686808 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:45.686774 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:45.687271 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:45.686775 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:45.687271 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:45.686910 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 00:59:45.687271 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:45.687008 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 00:59:46.687768 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:46.687569 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:46.688225 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:46.687866 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 00:59:47.687670 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:47.687625 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:47.687670 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:47.687671 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:47.688199 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:47.687773 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 00:59:47.688199 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:47.687907 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 00:59:48.366536 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:48.366492 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:48.366536 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:48.366546 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs\") pod \"network-metrics-daemon-kch6v\" (UID: \"3c724b42-2885-4687-970d-9ef08ddf10ff\") " pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:48.366779 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:48.366663 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 00:59:48.366779 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:48.366729 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret podName:f7953918-10f5-4a00-8b0a-d89348f25314 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:04.366710114 +0000 UTC m=+33.269565591 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret") pod "global-pull-secret-syncer-rmnsp" (UID: "f7953918-10f5-4a00-8b0a-d89348f25314") : object "kube-system"/"original-pull-secret" not registered Jul 14 00:59:48.366779 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:48.366663 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:59:48.366779 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:48.366779 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs podName:3c724b42-2885-4687-970d-9ef08ddf10ff nodeName:}" failed. No retries permitted until 2026-07-14 01:00:04.36676634 +0000 UTC m=+33.269621818 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs") pod "network-metrics-daemon-kch6v" (UID: "3c724b42-2885-4687-970d-9ef08ddf10ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:59:48.467628 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:48.467510 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7nq6\" (UniqueName: \"kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6\") pod \"network-check-target-rcg77\" (UID: \"84c558a6-a95a-44c3-8500-3e5d85a08a0a\") " pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:48.467805 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:48.467677 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 00:59:48.467805 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:48.467702 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 00:59:48.467805 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:48.467716 2582 projected.go:196] Error preparing data for projected volume kube-api-access-m7nq6 for pod openshift-network-diagnostics/network-check-target-rcg77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:59:48.467805 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:48.467783 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6 podName:84c558a6-a95a-44c3-8500-3e5d85a08a0a nodeName:}" failed. No retries permitted until 2026-07-14 01:00:04.467760526 +0000 UTC m=+33.370616026 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7nq6" (UniqueName: "kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6") pod "network-check-target-rcg77" (UID: "84c558a6-a95a-44c3-8500-3e5d85a08a0a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:59:48.687558 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:48.687486 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:48.687705 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:48.687595 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 00:59:49.687257 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:49.687221 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:49.687685 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:49.687270 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:49.687685 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:49.687384 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 00:59:49.687685 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:49.687501 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 00:59:50.687232 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:50.687194 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:50.687445 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:50.687349 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 00:59:51.688210 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:51.688165 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:51.688713 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:51.688299 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 00:59:51.688713 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:51.688471 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:51.688713 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:51.688589 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 00:59:52.686868 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:52.686830 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:52.687041 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:52.686960 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 00:59:53.687094 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:53.687061 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:53.687615 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:53.687105 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:53.687615 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:53.687197 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 00:59:53.687615 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:53.687286 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 00:59:53.913597 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:53.913368 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" event={"ID":"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f","Type":"ContainerStarted","Data":"d0c1c618ce2018cf607f85debf41c612fdf485133820ddb82f7657af01109173"} Jul 14 00:59:53.913737 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:53.913633 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" event={"ID":"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f","Type":"ContainerStarted","Data":"cddf4c96903bcd3d13a777a9ffe386e07fce7b2661976ad5767661349dc39eef"} Jul 14 00:59:53.920708 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:53.920649 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" event={"ID":"a7445fbc0edf80cb2d4f53f5ad08f1d5","Type":"ContainerStarted","Data":"6a5ac94e725723d27a896d4648a959772a8435a3d412a3adae38f8512a85bf9f"} Jul 14 00:59:53.922574 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:53.922546 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" event={"ID":"f3495fec-db67-437f-9803-3d2672f63e2e","Type":"ContainerStarted","Data":"e5f416b7b1afba6a725fee933fed93d7f5feb30db7776e5cb802b8d43f5abfd8"} Jul 14 00:59:53.924259 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:53.924115 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hfvn8" event={"ID":"7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8","Type":"ContainerStarted","Data":"6826465b55464228e48d30ad2b01b1323649fd63858bac39b0f2f712219f8213"} Jul 14 00:59:53.936645 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:53.936601 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-159.ec2.internal" podStartSLOduration=21.936586665 podStartE2EDuration="21.936586665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:59:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:59:53.935846082 +0000 UTC m=+22.838701582" watchObservedRunningTime="2026-07-14 00:59:53.936586665 +0000 UTC m=+22.839442164" Jul 14 00:59:53.951543 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:53.951495 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hfvn8" podStartSLOduration=2.504189231 podStartE2EDuration="22.951475074s" totalImagesPullingTime="20.447285843s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:59:53.951082943 +0000 UTC m=+22.853938446" watchObservedRunningTime="2026-07-14 00:59:53.951475074 +0000 UTC m=+22.854330573" Jul 14 00:59:54.687213 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.686934 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:54.687858 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:54.687232 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 00:59:54.928338 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.928283 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" event={"ID":"09a3f486-7a39-4e01-b913-dbbc3c3c3173","Type":"ContainerStarted","Data":"2066a9e11bca3ef639f077ec175cdbf0922646fff8ae6a921c373794387efbb6"} Jul 14 00:59:54.931693 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.931656 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" event={"ID":"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f","Type":"ContainerStarted","Data":"109d0122f55bec4c733cc03d5ce25aaa99149b7a94252d218e6e6dc8028fa1ab"} Jul 14 00:59:54.931836 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.931744 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" event={"ID":"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f","Type":"ContainerStarted","Data":"dfc4fd52d91b73bc8181bb5cc0194fb8acedd954e478b9437b6a79dfbbcca699"} Jul 14 00:59:54.931836 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.931794 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" event={"ID":"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f","Type":"ContainerStarted","Data":"167cd6d1dc6edef95258892722e4e77fea1f7392df2832af5de23f8827df22cd"} Jul 14 00:59:54.931957 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.931842 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" event={"ID":"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f","Type":"ContainerStarted","Data":"d4a26d67e002fa2578a658591e5a5a00722d7a4751c5070d6624050e94437de2"} Jul 14 00:59:54.933335 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.933301 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d4s6j" event={"ID":"6cc9c8a4-9f6d-481b-92e5-ae3e76527aa8","Type":"ContainerStarted","Data":"1fc4436983cc9a5922a55430b2c393deab4ea171c0b901fee4eac8e9ed087ec0"} Jul 14 00:59:54.935134 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.935111 2582 generic.go:356] "Generic (PLEG): container finished" podID="6a5c2793-deb1-4de9-953f-28ef081786c7" containerID="56ad9a6f74901eb4ca0b272c324e2125ba0a877e9e3014f2ccd5f302515054a9" exitCode=0 Jul 14 00:59:54.935236 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.935178 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7hspr" event={"ID":"6a5c2793-deb1-4de9-953f-28ef081786c7","Type":"ContainerDied","Data":"56ad9a6f74901eb4ca0b272c324e2125ba0a877e9e3014f2ccd5f302515054a9"} Jul 14 00:59:54.938067 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.937995 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-24x4q" event={"ID":"24004bce-6659-4fc1-a3e5-8a8d85103612","Type":"ContainerStarted","Data":"59febb402539c924fdcd5d1cfa6e2267c124941d1e1da0a7a35e0c2bdb096778"} Jul 14 00:59:54.939557 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.939532 2582 generic.go:356] "Generic (PLEG): container finished" podID="08020b785c0c8b6a8251ff1124de6164" containerID="d5178563330a8dad958f68a68ff1216933c88b4a8b8b5d8e9460783d2e331e77" exitCode=0 Jul 14 00:59:54.939662 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.939605 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" event={"ID":"08020b785c0c8b6a8251ff1124de6164","Type":"ContainerDied","Data":"d5178563330a8dad958f68a68ff1216933c88b4a8b8b5d8e9460783d2e331e77"} Jul 14 00:59:54.941237 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.941214 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pgcsp" event={"ID":"7d405593-7032-416c-852c-c947b5481d4f","Type":"ContainerStarted","Data":"c482ddf62c1c5a8e4eda25ef5328a9622093b2ce3ff63ac2e56eaee7b26b9d98"} Jul 14 00:59:54.942694 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.942632 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5hpkv" event={"ID":"c31e51b7-7bee-4ce8-9517-a1e6814a2641","Type":"ContainerStarted","Data":"14f6ca24f0477a9456718815efb8c7b7e806777833ad5701dce867cc5c0627b5"} Jul 14 00:59:54.947627 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.947580 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4pq4q" podStartSLOduration=3.588408006 podStartE2EDuration="23.947564407s" totalImagesPullingTime="20.359156401s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:59:53.966575879 +0000 UTC m=+22.869431378" watchObservedRunningTime="2026-07-14 00:59:54.947564407 +0000 UTC m=+23.850419905" Jul 14 00:59:54.947951 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.947921 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-d4s6j" podStartSLOduration=3.548536401 podStartE2EDuration="23.947912403s" totalImagesPullingTime="20.399376002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:59:54.947481248 +0000 UTC m=+23.850336747" watchObservedRunningTime="2026-07-14 00:59:54.947912403 +0000 UTC m=+23.850767903" Jul 14 00:59:54.974005 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:54.973958 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pgcsp" podStartSLOduration=4.001622611 podStartE2EDuration="23.973942217s" totalImagesPullingTime="19.972319606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:59:54.973740261 +0000 UTC m=+23.876595761" watchObservedRunningTime="2026-07-14 00:59:54.973942217 +0000 UTC m=+23.876797715" Jul 14 00:59:55.007865 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:55.007813 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5hpkv" podStartSLOduration=3.679564393 podStartE2EDuration="24.007800183s" totalImagesPullingTime="20.32823579s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:59:55.0074899 +0000 UTC m=+23.910345400" watchObservedRunningTime="2026-07-14 00:59:55.007800183 +0000 UTC m=+23.910655682" Jul 14 00:59:55.024463 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:55.024409 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-24x4q" podStartSLOduration=3.70291849 podStartE2EDuration="24.02439413s" totalImagesPullingTime="20.32147564s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:59:55.023868532 +0000 UTC m=+23.926724033" watchObservedRunningTime="2026-07-14 00:59:55.02439413 +0000 UTC m=+23.927249620" Jul 14 00:59:55.443069 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:55.443037 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 00:59:55.602401 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:55.602228 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T00:59:55.443064325Z","UUID":"0e9235d0-d86c-43c6-81e7-bc8c07b38b96","Handler":null,"Name":"","Endpoint":""} Jul 14 00:59:55.603923 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:55.603901 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 00:59:55.604075 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:55.603942 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 00:59:55.687428 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:55.687387 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:55.687863 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:55.687387 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:55.687863 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:55.687515 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 00:59:55.687863 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:55.687597 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 00:59:55.947753 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:55.947663 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" event={"ID":"08020b785c0c8b6a8251ff1124de6164","Type":"ContainerStarted","Data":"8539896482cf121fc79fd734cf2c9a22743790f97813f397b9973ab0a1731347"} Jul 14 00:59:55.949684 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:55.949648 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" event={"ID":"09a3f486-7a39-4e01-b913-dbbc3c3c3173","Type":"ContainerStarted","Data":"c7cfcc589516d8d132d8b0f721e110b1474b8007831169bbaafb6ebf1a4282fb"} Jul 14 00:59:55.961003 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:55.960957 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-159.ec2.internal" podStartSLOduration=23.960941257 podStartE2EDuration="23.960941257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:59:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:59:55.960752273 +0000 UTC m=+24.863607773" watchObservedRunningTime="2026-07-14 00:59:55.960941257 +0000 UTC m=+24.863796755" Jul 14 00:59:56.456723 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:56.456683 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pgcsp" Jul 14 00:59:56.457344 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:56.457301 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pgcsp" Jul 14 00:59:56.687630 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:56.687120 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:56.687630 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:56.687260 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 00:59:56.956298 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:56.956265 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" event={"ID":"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f","Type":"ContainerStarted","Data":"40b06a303172bbf1b0de46a94b727f71194ba5096926b6880aaa2bfbe8517d10"} Jul 14 00:59:56.956761 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:56.956739 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pgcsp" Jul 14 00:59:56.957248 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:56.957233 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pgcsp" Jul 14 00:59:57.687120 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:57.687081 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:57.687306 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:57.687215 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 00:59:57.687306 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:57.687267 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:57.687450 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:57.687380 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 00:59:57.961175 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:57.961093 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" event={"ID":"09a3f486-7a39-4e01-b913-dbbc3c3c3173","Type":"ContainerStarted","Data":"0c124d21c3493a8dad850cbaa6ae2efe26bf5fcddc0814f09e88e46da8f32077"} Jul 14 00:59:57.977469 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:57.977412 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bvswz" podStartSLOduration=3.795205677 podStartE2EDuration="26.9773956s" totalImagesPullingTime="23.182189923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:59:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:59:57.977020093 +0000 UTC m=+26.879875596" watchObservedRunningTime="2026-07-14 00:59:57.9773956 +0000 UTC m=+26.880251099" Jul 14 00:59:58.687162 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:58.687123 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 00:59:58.687357 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:58.687293 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 00:59:59.687309 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:59.687102 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 00:59:59.687965 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:59.687169 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 00:59:59.687965 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:59.687356 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 00:59:59.687965 ip-10-0-143-159 kubenswrapper[2582]: E0714 00:59:59.687438 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 00:59:59.968905 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:59.968827 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" event={"ID":"3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f","Type":"ContainerStarted","Data":"968560c6bb65a235538190ee0789a157facc81fc2f72163bae65e355fba30ba4"} Jul 14 00:59:59.969089 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:59.969072 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:59.970686 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:59.970663 2582 generic.go:356] "Generic (PLEG): container finished" podID="6a5c2793-deb1-4de9-953f-28ef081786c7" containerID="8700d240638ce9d5c9cd6004703421d27c7d43baf81c64ef71a49e27f0538618" exitCode=0 Jul 14 00:59:59.970795 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:59.970702 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7hspr" event={"ID":"6a5c2793-deb1-4de9-953f-28ef081786c7","Type":"ContainerDied","Data":"8700d240638ce9d5c9cd6004703421d27c7d43baf81c64ef71a49e27f0538618"} Jul 14 00:59:59.985456 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:59.985434 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 00:59:59.995522 ip-10-0-143-159 kubenswrapper[2582]: I0714 00:59:59.995484 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" podStartSLOduration=8.266350546 podStartE2EDuration="28.99547146s" totalImagesPullingTime="20.729120914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:59:59.994871575 +0000 UTC m=+28.897727074" watchObservedRunningTime="2026-07-14 00:59:59.99547146 +0000 UTC m=+28.898326959" Jul 14 01:00:00.686670 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:00.686491 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 01:00:00.686920 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:00.686762 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 01:00:00.974448 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:00.972842 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 01:00:00.974877 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:00.974474 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 01:00:00.991887 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:00.991865 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 01:00:01.014383 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:01.014356 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rmnsp"] Jul 14 01:00:01.014505 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:01.014486 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 01:00:01.014631 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:01.014604 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 01:00:01.019229 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:01.019205 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rcg77"] Jul 14 01:00:01.019553 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:01.019438 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kch6v"] Jul 14 01:00:01.019553 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:01.019532 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 01:00:01.019719 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:01.019644 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 01:00:01.019719 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:01.019706 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 01:00:01.019861 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:01.019775 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 01:00:01.977035 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:01.976997 2582 generic.go:356] "Generic (PLEG): container finished" podID="6a5c2793-deb1-4de9-953f-28ef081786c7" containerID="18db179134f6be57503e1d2077c6078dc972909a64fe5fb6b8a0d9547db3a328" exitCode=0 Jul 14 01:00:01.977453 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:01.977083 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7hspr" event={"ID":"6a5c2793-deb1-4de9-953f-28ef081786c7","Type":"ContainerDied","Data":"18db179134f6be57503e1d2077c6078dc972909a64fe5fb6b8a0d9547db3a328"} Jul 14 01:00:02.687534 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:02.687497 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 01:00:02.687709 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:02.687497 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 01:00:02.687709 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:02.687607 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 01:00:02.687792 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:02.687700 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 01:00:02.687792 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:02.687497 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 01:00:02.687866 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:02.687834 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 01:00:03.984064 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:03.983888 2582 generic.go:356] "Generic (PLEG): container finished" podID="6a5c2793-deb1-4de9-953f-28ef081786c7" containerID="5a7cc074fe48e8efce8d6e7603f8d02cb2ea8b69d46a955c58ef4192d07d8f30" exitCode=0 Jul 14 01:00:03.984064 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:03.983964 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7hspr" event={"ID":"6a5c2793-deb1-4de9-953f-28ef081786c7","Type":"ContainerDied","Data":"5a7cc074fe48e8efce8d6e7603f8d02cb2ea8b69d46a955c58ef4192d07d8f30"} Jul 14 01:00:04.381102 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:04.381068 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs\") pod \"network-metrics-daemon-kch6v\" (UID: \"3c724b42-2885-4687-970d-9ef08ddf10ff\") " pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 01:00:04.381264 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:04.381128 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 01:00:04.381264 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:04.381218 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 01:00:04.381264 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:04.381234 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 01:00:04.381406 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:04.381271 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret podName:f7953918-10f5-4a00-8b0a-d89348f25314 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:36.381256911 +0000 UTC m=+65.284112388 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret") pod "global-pull-secret-syncer-rmnsp" (UID: "f7953918-10f5-4a00-8b0a-d89348f25314") : object "kube-system"/"original-pull-secret" not registered Jul 14 01:00:04.381406 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:04.381301 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs podName:3c724b42-2885-4687-970d-9ef08ddf10ff nodeName:}" failed. No retries permitted until 2026-07-14 01:00:36.381283198 +0000 UTC m=+65.284138695 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs") pod "network-metrics-daemon-kch6v" (UID: "3c724b42-2885-4687-970d-9ef08ddf10ff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 01:00:04.481616 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:04.481590 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7nq6\" (UniqueName: \"kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6\") pod \"network-check-target-rcg77\" (UID: \"84c558a6-a95a-44c3-8500-3e5d85a08a0a\") " pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 01:00:04.481762 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:04.481719 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 01:00:04.481762 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:04.481739 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 01:00:04.481762 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:04.481748 2582 projected.go:196] Error preparing data for projected volume kube-api-access-m7nq6 for pod openshift-network-diagnostics/network-check-target-rcg77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 01:00:04.481874 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:04.481799 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6 podName:84c558a6-a95a-44c3-8500-3e5d85a08a0a nodeName:}" failed. No retries permitted until 2026-07-14 01:00:36.481783885 +0000 UTC m=+65.384639362 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7nq6" (UniqueName: "kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6") pod "network-check-target-rcg77" (UID: "84c558a6-a95a-44c3-8500-3e5d85a08a0a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 01:00:04.686861 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:04.686774 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 01:00:04.687035 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:04.686908 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rmnsp" podUID="f7953918-10f5-4a00-8b0a-d89348f25314" Jul 14 01:00:04.687300 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:04.687280 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 01:00:04.687443 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:04.687413 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kch6v" podUID="3c724b42-2885-4687-970d-9ef08ddf10ff" Jul 14 01:00:04.687507 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:04.687490 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 01:00:04.687576 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:04.687559 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcg77" podUID="84c558a6-a95a-44c3-8500-3e5d85a08a0a" Jul 14 01:00:06.366015 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.365983 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-159.ec2.internal" event="NodeReady" Jul 14 01:00:06.366437 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.366117 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 01:00:06.414747 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.414707 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jcbh7"] Jul 14 01:00:06.417488 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.417461 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6x4hg"] Jul 14 01:00:06.417615 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.417596 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:06.420232 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.419948 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6x4hg" Jul 14 01:00:06.420723 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.420588 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 01:00:06.420723 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.420610 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 01:00:06.420723 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.420624 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x8vr9\"" Jul 14 01:00:06.422156 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.422136 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 01:00:06.422280 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.422189 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 01:00:06.422368 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.422308 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 01:00:06.422493 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.422469 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fbz48\"" Jul 14 01:00:06.426428 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.426357 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6x4hg"] Jul 14 01:00:06.428861 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.428840 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jcbh7"] Jul 14 01:00:06.493810 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.493773 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2342051-9f5f-4acc-bfd9-67d46c0d9389-tmp-dir\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:06.493810 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.493813 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mxgj\" (UniqueName: \"kubernetes.io/projected/1191145f-7591-45da-9a24-7c20fc700c59-kube-api-access-5mxgj\") pod \"ingress-canary-6x4hg\" (UID: \"1191145f-7591-45da-9a24-7c20fc700c59\") " pod="openshift-ingress-canary/ingress-canary-6x4hg" Jul 14 01:00:06.494049 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.493890 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1191145f-7591-45da-9a24-7c20fc700c59-cert\") pod \"ingress-canary-6x4hg\" (UID: \"1191145f-7591-45da-9a24-7c20fc700c59\") " pod="openshift-ingress-canary/ingress-canary-6x4hg" Jul 14 01:00:06.494049 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.493980 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a2342051-9f5f-4acc-bfd9-67d46c0d9389-config-volume\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:06.494049 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.494016 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctw7j\" (UniqueName: \"kubernetes.io/projected/a2342051-9f5f-4acc-bfd9-67d46c0d9389-kube-api-access-ctw7j\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:06.494206 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.494122 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:06.594660 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.594626 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a2342051-9f5f-4acc-bfd9-67d46c0d9389-config-volume\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:06.594660 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.594663 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctw7j\" (UniqueName: \"kubernetes.io/projected/a2342051-9f5f-4acc-bfd9-67d46c0d9389-kube-api-access-ctw7j\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:06.594891 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.594703 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:06.594891 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.594749 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2342051-9f5f-4acc-bfd9-67d46c0d9389-tmp-dir\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:06.594891 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.594773 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mxgj\" (UniqueName: \"kubernetes.io/projected/1191145f-7591-45da-9a24-7c20fc700c59-kube-api-access-5mxgj\") pod \"ingress-canary-6x4hg\" (UID: \"1191145f-7591-45da-9a24-7c20fc700c59\") " pod="openshift-ingress-canary/ingress-canary-6x4hg" Jul 14 01:00:06.594891 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.594809 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1191145f-7591-45da-9a24-7c20fc700c59-cert\") pod \"ingress-canary-6x4hg\" (UID: \"1191145f-7591-45da-9a24-7c20fc700c59\") " pod="openshift-ingress-canary/ingress-canary-6x4hg" Jul 14 01:00:06.595075 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:06.594898 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 01:00:06.595075 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:06.594984 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls podName:a2342051-9f5f-4acc-bfd9-67d46c0d9389 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:07.094958699 +0000 UTC m=+35.997814199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls") pod "dns-default-jcbh7" (UID: "a2342051-9f5f-4acc-bfd9-67d46c0d9389") : secret "dns-default-metrics-tls" not found Jul 14 01:00:06.595254 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.595227 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2342051-9f5f-4acc-bfd9-67d46c0d9389-tmp-dir\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:06.595368 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.595347 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a2342051-9f5f-4acc-bfd9-67d46c0d9389-config-volume\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:06.599924 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.599898 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1191145f-7591-45da-9a24-7c20fc700c59-cert\") pod \"ingress-canary-6x4hg\" (UID: \"1191145f-7591-45da-9a24-7c20fc700c59\") " pod="openshift-ingress-canary/ingress-canary-6x4hg" Jul 14 01:00:06.603212 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.603188 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctw7j\" (UniqueName: \"kubernetes.io/projected/a2342051-9f5f-4acc-bfd9-67d46c0d9389-kube-api-access-ctw7j\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:06.603512 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.603491 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mxgj\" (UniqueName: \"kubernetes.io/projected/1191145f-7591-45da-9a24-7c20fc700c59-kube-api-access-5mxgj\") pod \"ingress-canary-6x4hg\" (UID: \"1191145f-7591-45da-9a24-7c20fc700c59\") " pod="openshift-ingress-canary/ingress-canary-6x4hg" Jul 14 01:00:06.687673 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.687587 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 01:00:06.687844 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.687587 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 01:00:06.687844 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.687731 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 01:00:06.690569 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.690544 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 01:00:06.690569 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.690561 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 01:00:06.690569 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.690571 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 01:00:06.690788 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.690569 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 01:00:06.690788 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.690668 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hm62n\"" Jul 14 01:00:06.690919 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.690877 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-btv6j\"" Jul 14 01:00:06.739929 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:06.739896 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6x4hg" Jul 14 01:00:07.099555 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:07.099524 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:07.099738 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:07.099702 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 01:00:07.099796 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:07.099775 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls podName:a2342051-9f5f-4acc-bfd9-67d46c0d9389 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:08.099758426 +0000 UTC m=+37.002613903 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls") pod "dns-default-jcbh7" (UID: "a2342051-9f5f-4acc-bfd9-67d46c0d9389") : secret "dns-default-metrics-tls" not found Jul 14 01:00:08.108035 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:08.108000 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:08.108711 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:08.108144 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 01:00:08.108711 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:08.108217 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls podName:a2342051-9f5f-4acc-bfd9-67d46c0d9389 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:10.108199834 +0000 UTC m=+39.011055311 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls") pod "dns-default-jcbh7" (UID: "a2342051-9f5f-4acc-bfd9-67d46c0d9389") : secret "dns-default-metrics-tls" not found Jul 14 01:00:08.280527 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:08.280374 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6x4hg"] Jul 14 01:00:08.287381 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:00:08.287349 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1191145f_7591_45da_9a24_7c20fc700c59.slice/crio-7bbd69f42523ab32b0f063749fea4a80e508760f17f1f76e0326c12455f344ae WatchSource:0}: Error finding container 7bbd69f42523ab32b0f063749fea4a80e508760f17f1f76e0326c12455f344ae: Status 404 returned error can't find the container with id 7bbd69f42523ab32b0f063749fea4a80e508760f17f1f76e0326c12455f344ae Jul 14 01:00:08.997238 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:08.997197 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6x4hg" event={"ID":"1191145f-7591-45da-9a24-7c20fc700c59","Type":"ContainerStarted","Data":"7bbd69f42523ab32b0f063749fea4a80e508760f17f1f76e0326c12455f344ae"} Jul 14 01:00:08.999992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:08.999971 2582 generic.go:356] "Generic (PLEG): container finished" podID="6a5c2793-deb1-4de9-953f-28ef081786c7" containerID="6b2a18359cf869431e9bf9aaf48c37e3f10c8bd6a20c053d09c05be4a0412709" exitCode=0 Jul 14 01:00:09.000054 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:09.000026 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7hspr" event={"ID":"6a5c2793-deb1-4de9-953f-28ef081786c7","Type":"ContainerDied","Data":"6b2a18359cf869431e9bf9aaf48c37e3f10c8bd6a20c053d09c05be4a0412709"} Jul 14 01:00:10.007928 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:10.007887 2582 generic.go:356] "Generic (PLEG): container finished" podID="6a5c2793-deb1-4de9-953f-28ef081786c7" containerID="75f4c3c07e25a086fa5b31e450459cc2a0154c3bb4e08c0b247266a9cc046342" exitCode=0 Jul 14 01:00:10.008651 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:10.007953 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7hspr" event={"ID":"6a5c2793-deb1-4de9-953f-28ef081786c7","Type":"ContainerDied","Data":"75f4c3c07e25a086fa5b31e450459cc2a0154c3bb4e08c0b247266a9cc046342"} Jul 14 01:00:10.122100 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:10.122061 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:10.122286 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:10.122212 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 01:00:10.122286 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:10.122284 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls podName:a2342051-9f5f-4acc-bfd9-67d46c0d9389 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:14.122263643 +0000 UTC m=+43.025119140 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls") pod "dns-default-jcbh7" (UID: "a2342051-9f5f-4acc-bfd9-67d46c0d9389") : secret "dns-default-metrics-tls" not found Jul 14 01:00:11.013339 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:11.013243 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7hspr" event={"ID":"6a5c2793-deb1-4de9-953f-28ef081786c7","Type":"ContainerStarted","Data":"5159ab9abbe1628d7f52486aa62e226057ed67f8426afe581e955ec5b995bd0f"} Jul 14 01:00:11.034669 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:11.034621 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7hspr" podStartSLOduration=9.048505046 podStartE2EDuration="40.034605364s" totalImagesPullingTime="30.986100318s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:59:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:11.03312822 +0000 UTC m=+39.935983720" watchObservedRunningTime="2026-07-14 01:00:11.034605364 +0000 UTC m=+39.937460863" Jul 14 01:00:12.017027 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:12.016986 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6x4hg" event={"ID":"1191145f-7591-45da-9a24-7c20fc700c59","Type":"ContainerStarted","Data":"97ae4b726c9c76ff387a63d51a0f0697b767e609d375964890bb0e9e3819c666"} Jul 14 01:00:12.032979 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:12.032850 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6x4hg" podStartSLOduration=3.009941576 podStartE2EDuration="6.032832647s" totalImagesPullingTime="3.022891071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:12.032218265 +0000 UTC m=+40.935073764" watchObservedRunningTime="2026-07-14 01:00:12.032832647 +0000 UTC m=+40.935688147" Jul 14 01:00:13.651629 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.651599 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-8cpht"] Jul 14 01:00:13.676888 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.676857 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4"] Jul 14 01:00:13.677019 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.676980 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8cpht" Jul 14 01:00:13.679566 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.679539 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-q5wcl\"" Jul 14 01:00:13.702815 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.702790 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-tvhvj"] Jul 14 01:00:13.702947 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.702899 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" Jul 14 01:00:13.705166 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.705140 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 14 01:00:13.705283 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.705169 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 14 01:00:13.705407 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.705390 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 14 01:00:13.705493 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.705478 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 14 01:00:13.705649 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.705631 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-7fshn\"" Jul 14 01:00:13.724966 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.724943 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.727797 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.727773 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 14 01:00:13.727797 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.727795 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 01:00:13.728028 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.727783 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 14 01:00:13.728028 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.727846 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-gmrqg\"" Jul 14 01:00:13.728160 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.728144 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 01:00:13.728694 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.728676 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-8cpht"] Jul 14 01:00:13.728806 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.728721 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4"] Jul 14 01:00:13.728806 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.728745 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-tvhvj"] Jul 14 01:00:13.732743 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.732722 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 14 01:00:13.748375 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.748346 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjdx2\" (UniqueName: \"kubernetes.io/projected/83eeb05b-795c-487c-9714-6c1deba18873-kube-api-access-jjdx2\") pod \"network-check-source-869dcfcdcd-8cpht\" (UID: \"83eeb05b-795c-487c-9714-6c1deba18873\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8cpht" Jul 14 01:00:13.756060 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.756040 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7jgwz"] Jul 14 01:00:13.778912 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.778884 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm"] Jul 14 01:00:13.779069 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.779027 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7jgwz" Jul 14 01:00:13.781844 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.781818 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 14 01:00:13.781964 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.781818 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 14 01:00:13.781964 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.781917 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-5zf9b\"" Jul 14 01:00:13.805408 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.805381 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4"] Jul 14 01:00:13.805555 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.805504 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" Jul 14 01:00:13.809429 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.809407 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 14 01:00:13.809557 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.809493 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-zbq6l\"" Jul 14 01:00:13.809914 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.809892 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 14 01:00:13.810029 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.810004 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 14 01:00:13.810111 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.810096 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 14 01:00:13.828153 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.828122 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-wgj6h"] Jul 14 01:00:13.828285 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.828237 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" Jul 14 01:00:13.830639 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.830619 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 14 01:00:13.830747 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.830645 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-llwd9\"" Jul 14 01:00:13.830747 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.830621 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 14 01:00:13.830946 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.830930 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 14 01:00:13.842164 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.842137 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7jgwz"] Jul 14 01:00:13.842263 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.842172 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm"] Jul 14 01:00:13.842263 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.842193 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-wgj6h"] Jul 14 01:00:13.842263 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.842211 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4"] Jul 14 01:00:13.842263 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.842258 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:13.845220 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.845184 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 14 01:00:13.845220 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.845202 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 14 01:00:13.845220 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.845206 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 14 01:00:13.845220 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.845205 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 14 01:00:13.845492 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.845206 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-wzftr\"" Jul 14 01:00:13.848817 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.848796 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/be11544a-5834-4744-a9d6-a6bd32372ee6-snapshots\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.848972 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.848844 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be11544a-5834-4744-a9d6-a6bd32372ee6-tmp\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.848972 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.848874 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be11544a-5834-4744-a9d6-a6bd32372ee6-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.849083 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.848986 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2f72\" (UniqueName: \"kubernetes.io/projected/9c7799c6-a631-4555-b405-cbcf44b861ce-kube-api-access-h2f72\") pod \"kube-storage-version-migrator-operator-799cb7f958-74cl4\" (UID: \"9c7799c6-a631-4555-b405-cbcf44b861ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" Jul 14 01:00:13.849083 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.849039 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9c7799c6-a631-4555-b405-cbcf44b861ce-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-74cl4\" (UID: \"9c7799c6-a631-4555-b405-cbcf44b861ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" Jul 14 01:00:13.849083 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.849061 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjdx2\" (UniqueName: \"kubernetes.io/projected/83eeb05b-795c-487c-9714-6c1deba18873-kube-api-access-jjdx2\") pod \"network-check-source-869dcfcdcd-8cpht\" (UID: \"83eeb05b-795c-487c-9714-6c1deba18873\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8cpht" Jul 14 01:00:13.849238 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.849127 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be11544a-5834-4744-a9d6-a6bd32372ee6-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.849238 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.849166 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be11544a-5834-4744-a9d6-a6bd32372ee6-serving-cert\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.849238 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.849209 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqm5p\" (UniqueName: \"kubernetes.io/projected/be11544a-5834-4744-a9d6-a6bd32372ee6-kube-api-access-rqm5p\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.849238 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.849231 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9c7799c6-a631-4555-b405-cbcf44b861ce-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-74cl4\" (UID: \"9c7799c6-a631-4555-b405-cbcf44b861ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" Jul 14 01:00:13.849418 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.849264 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlchv\" (UniqueName: \"kubernetes.io/projected/791300b3-0b23-48c5-84a8-3b9b52954625-kube-api-access-nlchv\") pod \"volume-data-source-validator-78495597f6-7jgwz\" (UID: \"791300b3-0b23-48c5-84a8-3b9b52954625\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7jgwz" Jul 14 01:00:13.850080 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.850061 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 14 01:00:13.855194 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.855175 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-646d9864b5-pt6rn"] Jul 14 01:00:13.868780 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.868754 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjdx2\" (UniqueName: \"kubernetes.io/projected/83eeb05b-795c-487c-9714-6c1deba18873-kube-api-access-jjdx2\") pod \"network-check-source-869dcfcdcd-8cpht\" (UID: \"83eeb05b-795c-487c-9714-6c1deba18873\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8cpht" Jul 14 01:00:13.871442 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.871423 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:13.871610 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.871583 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-646d9864b5-pt6rn"] Jul 14 01:00:13.874074 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.874055 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gn7kr\"" Jul 14 01:00:13.874074 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.874129 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 14 01:00:13.874074 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.874148 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 14 01:00:13.874074 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.874154 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 14 01:00:13.883914 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.883895 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 14 01:00:13.949897 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.949798 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqm5p\" (UniqueName: \"kubernetes.io/projected/be11544a-5834-4744-a9d6-a6bd32372ee6-kube-api-access-rqm5p\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.949897 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.949841 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-np8n4\" (UID: \"ccb9e582-0ff4-40af-8c5b-38ffe27dea5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" Jul 14 01:00:13.949897 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.949867 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/47f717ea-28d1-474f-8c21-1742d49d4306-trusted-ca\") pod \"console-operator-7866497494-wgj6h\" (UID: \"47f717ea-28d1-474f-8c21-1742d49d4306\") " pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:13.949897 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.949883 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be11544a-5834-4744-a9d6-a6bd32372ee6-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.949897 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.949901 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-image-registry-private-configuration\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:13.950185 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950012 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-certificates\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:13.950185 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950050 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-installation-pull-secrets\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:13.950185 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950081 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/47f717ea-28d1-474f-8c21-1742d49d4306-config\") pod \"console-operator-7866497494-wgj6h\" (UID: \"47f717ea-28d1-474f-8c21-1742d49d4306\") " pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:13.950185 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950107 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-trusted-ca\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:13.950185 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950136 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be11544a-5834-4744-a9d6-a6bd32372ee6-tmp\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.950370 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950223 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/47f717ea-28d1-474f-8c21-1742d49d4306-serving-cert\") pod \"console-operator-7866497494-wgj6h\" (UID: \"47f717ea-28d1-474f-8c21-1742d49d4306\") " pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:13.950370 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950252 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-bound-sa-token\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:13.950370 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950273 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16d054d1-dce4-479b-9adc-04f7b4feb9f9-config\") pod \"service-ca-operator-7695fc6c8c-42xrm\" (UID: \"16d054d1-dce4-479b-9adc-04f7b4feb9f9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" Jul 14 01:00:13.950454 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950391 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9c7799c6-a631-4555-b405-cbcf44b861ce-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-74cl4\" (UID: \"9c7799c6-a631-4555-b405-cbcf44b861ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" Jul 14 01:00:13.950454 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950430 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:13.950528 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950456 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-ca-trust-extracted\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:13.950528 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950498 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5cl2\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-kube-api-access-p5cl2\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:13.950528 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950510 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be11544a-5834-4744-a9d6-a6bd32372ee6-tmp\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.950619 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950528 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9c7799c6-a631-4555-b405-cbcf44b861ce-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-74cl4\" (UID: \"9c7799c6-a631-4555-b405-cbcf44b861ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" Jul 14 01:00:13.950619 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950559 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlchv\" (UniqueName: \"kubernetes.io/projected/791300b3-0b23-48c5-84a8-3b9b52954625-kube-api-access-nlchv\") pod \"volume-data-source-validator-78495597f6-7jgwz\" (UID: \"791300b3-0b23-48c5-84a8-3b9b52954625\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7jgwz" Jul 14 01:00:13.950619 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950583 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkr7j\" (UniqueName: \"kubernetes.io/projected/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-kube-api-access-pkr7j\") pod \"cluster-samples-operator-65db858894-np8n4\" (UID: \"ccb9e582-0ff4-40af-8c5b-38ffe27dea5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" Jul 14 01:00:13.950619 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950607 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxr29\" (UniqueName: \"kubernetes.io/projected/47f717ea-28d1-474f-8c21-1742d49d4306-kube-api-access-qxr29\") pod \"console-operator-7866497494-wgj6h\" (UID: \"47f717ea-28d1-474f-8c21-1742d49d4306\") " pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:13.950761 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950636 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/be11544a-5834-4744-a9d6-a6bd32372ee6-snapshots\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.950761 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950689 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16d054d1-dce4-479b-9adc-04f7b4feb9f9-serving-cert\") pod \"service-ca-operator-7695fc6c8c-42xrm\" (UID: \"16d054d1-dce4-479b-9adc-04f7b4feb9f9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" Jul 14 01:00:13.950761 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950737 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be11544a-5834-4744-a9d6-a6bd32372ee6-serving-cert\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.950905 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950786 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be11544a-5834-4744-a9d6-a6bd32372ee6-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.950905 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950883 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2f72\" (UniqueName: \"kubernetes.io/projected/9c7799c6-a631-4555-b405-cbcf44b861ce-kube-api-access-h2f72\") pod \"kube-storage-version-migrator-operator-799cb7f958-74cl4\" (UID: \"9c7799c6-a631-4555-b405-cbcf44b861ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" Jul 14 01:00:13.950998 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950919 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc5xz\" (UniqueName: \"kubernetes.io/projected/16d054d1-dce4-479b-9adc-04f7b4feb9f9-kube-api-access-cc5xz\") pod \"service-ca-operator-7695fc6c8c-42xrm\" (UID: \"16d054d1-dce4-479b-9adc-04f7b4feb9f9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" Jul 14 01:00:13.950998 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.950934 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be11544a-5834-4744-a9d6-a6bd32372ee6-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.951117 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.951094 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9c7799c6-a631-4555-b405-cbcf44b861ce-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-74cl4\" (UID: \"9c7799c6-a631-4555-b405-cbcf44b861ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" Jul 14 01:00:13.951308 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.951279 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be11544a-5834-4744-a9d6-a6bd32372ee6-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.951461 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.951441 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/be11544a-5834-4744-a9d6-a6bd32372ee6-snapshots\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.952815 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.952796 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9c7799c6-a631-4555-b405-cbcf44b861ce-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-74cl4\" (UID: \"9c7799c6-a631-4555-b405-cbcf44b861ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" Jul 14 01:00:13.953081 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.953062 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be11544a-5834-4744-a9d6-a6bd32372ee6-serving-cert\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.958055 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.958031 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqm5p\" (UniqueName: \"kubernetes.io/projected/be11544a-5834-4744-a9d6-a6bd32372ee6-kube-api-access-rqm5p\") pod \"insights-operator-68d9dd94cf-tvhvj\" (UID: \"be11544a-5834-4744-a9d6-a6bd32372ee6\") " pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:13.958487 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.958464 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2f72\" (UniqueName: \"kubernetes.io/projected/9c7799c6-a631-4555-b405-cbcf44b861ce-kube-api-access-h2f72\") pod \"kube-storage-version-migrator-operator-799cb7f958-74cl4\" (UID: \"9c7799c6-a631-4555-b405-cbcf44b861ce\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" Jul 14 01:00:13.958769 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.958748 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlchv\" (UniqueName: \"kubernetes.io/projected/791300b3-0b23-48c5-84a8-3b9b52954625-kube-api-access-nlchv\") pod \"volume-data-source-validator-78495597f6-7jgwz\" (UID: \"791300b3-0b23-48c5-84a8-3b9b52954625\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7jgwz" Jul 14 01:00:13.985515 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:13.985481 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8cpht" Jul 14 01:00:14.010796 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.010760 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" Jul 14 01:00:14.033805 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.033777 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" Jul 14 01:00:14.051793 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.051759 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-np8n4\" (UID: \"ccb9e582-0ff4-40af-8c5b-38ffe27dea5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" Jul 14 01:00:14.051793 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.051800 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/47f717ea-28d1-474f-8c21-1742d49d4306-trusted-ca\") pod \"console-operator-7866497494-wgj6h\" (UID: \"47f717ea-28d1-474f-8c21-1742d49d4306\") " pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:14.052045 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.051827 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-image-registry-private-configuration\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.052045 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.051856 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-certificates\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.052045 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.051879 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-installation-pull-secrets\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.052045 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.051901 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/47f717ea-28d1-474f-8c21-1742d49d4306-config\") pod \"console-operator-7866497494-wgj6h\" (UID: \"47f717ea-28d1-474f-8c21-1742d49d4306\") " pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:14.052045 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:14.051907 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 01:00:14.052045 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.051925 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-trusted-ca\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.052045 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.051967 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/47f717ea-28d1-474f-8c21-1742d49d4306-serving-cert\") pod \"console-operator-7866497494-wgj6h\" (UID: \"47f717ea-28d1-474f-8c21-1742d49d4306\") " pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:14.052045 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:14.051989 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls podName:ccb9e582-0ff4-40af-8c5b-38ffe27dea5b nodeName:}" failed. No retries permitted until 2026-07-14 01:00:14.551967484 +0000 UTC m=+43.454822982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls") pod "cluster-samples-operator-65db858894-np8n4" (UID: "ccb9e582-0ff4-40af-8c5b-38ffe27dea5b") : secret "samples-operator-tls" not found Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.052963 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/47f717ea-28d1-474f-8c21-1742d49d4306-trusted-ca\") pod \"console-operator-7866497494-wgj6h\" (UID: \"47f717ea-28d1-474f-8c21-1742d49d4306\") " pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.053018 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-certificates\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.053075 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-bound-sa-token\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.053105 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16d054d1-dce4-479b-9adc-04f7b4feb9f9-config\") pod \"service-ca-operator-7695fc6c8c-42xrm\" (UID: \"16d054d1-dce4-479b-9adc-04f7b4feb9f9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.053160 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.053183 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-ca-trust-extracted\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.053211 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5cl2\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-kube-api-access-p5cl2\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.053239 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkr7j\" (UniqueName: \"kubernetes.io/projected/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-kube-api-access-pkr7j\") pod \"cluster-samples-operator-65db858894-np8n4\" (UID: \"ccb9e582-0ff4-40af-8c5b-38ffe27dea5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.053263 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxr29\" (UniqueName: \"kubernetes.io/projected/47f717ea-28d1-474f-8c21-1742d49d4306-kube-api-access-qxr29\") pod \"console-operator-7866497494-wgj6h\" (UID: \"47f717ea-28d1-474f-8c21-1742d49d4306\") " pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.053307 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16d054d1-dce4-479b-9adc-04f7b4feb9f9-serving-cert\") pod \"service-ca-operator-7695fc6c8c-42xrm\" (UID: \"16d054d1-dce4-479b-9adc-04f7b4feb9f9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.053386 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc5xz\" (UniqueName: \"kubernetes.io/projected/16d054d1-dce4-479b-9adc-04f7b4feb9f9-kube-api-access-cc5xz\") pod \"service-ca-operator-7695fc6c8c-42xrm\" (UID: \"16d054d1-dce4-479b-9adc-04f7b4feb9f9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.053538 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/47f717ea-28d1-474f-8c21-1742d49d4306-config\") pod \"console-operator-7866497494-wgj6h\" (UID: \"47f717ea-28d1-474f-8c21-1742d49d4306\") " pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.053646 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-ca-trust-extracted\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.053812 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-trusted-ca\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:14.053928 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 01:00:14.053992 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:14.053943 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-646d9864b5-pt6rn: secret "image-registry-tls" not found Jul 14 01:00:14.054875 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:14.054000 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls podName:d1310c00-61a2-4b6a-b23c-7740b8c8bdf0 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:14.553983152 +0000 UTC m=+43.456838645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls") pod "image-registry-646d9864b5-pt6rn" (UID: "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0") : secret "image-registry-tls" not found Jul 14 01:00:14.054875 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.054265 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16d054d1-dce4-479b-9adc-04f7b4feb9f9-config\") pod \"service-ca-operator-7695fc6c8c-42xrm\" (UID: \"16d054d1-dce4-479b-9adc-04f7b4feb9f9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" Jul 14 01:00:14.054875 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.054790 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/47f717ea-28d1-474f-8c21-1742d49d4306-serving-cert\") pod \"console-operator-7866497494-wgj6h\" (UID: \"47f717ea-28d1-474f-8c21-1742d49d4306\") " pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:14.054993 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.054909 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-image-registry-private-configuration\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.056123 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.056100 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-installation-pull-secrets\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.056420 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.056401 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16d054d1-dce4-479b-9adc-04f7b4feb9f9-serving-cert\") pod \"service-ca-operator-7695fc6c8c-42xrm\" (UID: \"16d054d1-dce4-479b-9adc-04f7b4feb9f9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" Jul 14 01:00:14.064277 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.064251 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5cl2\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-kube-api-access-p5cl2\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.065048 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.065022 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-bound-sa-token\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.065177 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.065157 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxr29\" (UniqueName: \"kubernetes.io/projected/47f717ea-28d1-474f-8c21-1742d49d4306-kube-api-access-qxr29\") pod \"console-operator-7866497494-wgj6h\" (UID: \"47f717ea-28d1-474f-8c21-1742d49d4306\") " pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:14.066884 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.066867 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkr7j\" (UniqueName: \"kubernetes.io/projected/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-kube-api-access-pkr7j\") pod \"cluster-samples-operator-65db858894-np8n4\" (UID: \"ccb9e582-0ff4-40af-8c5b-38ffe27dea5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" Jul 14 01:00:14.066944 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.066920 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc5xz\" (UniqueName: \"kubernetes.io/projected/16d054d1-dce4-479b-9adc-04f7b4feb9f9-kube-api-access-cc5xz\") pod \"service-ca-operator-7695fc6c8c-42xrm\" (UID: \"16d054d1-dce4-479b-9adc-04f7b4feb9f9\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" Jul 14 01:00:14.088466 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.088422 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7jgwz" Jul 14 01:00:14.114081 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.113986 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" Jul 14 01:00:14.151608 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.151579 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:14.154041 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.154019 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:14.154187 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:14.154173 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 01:00:14.154269 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:14.154222 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls podName:a2342051-9f5f-4acc-bfd9-67d46c0d9389 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:22.154208054 +0000 UTC m=+51.057063532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls") pod "dns-default-jcbh7" (UID: "a2342051-9f5f-4acc-bfd9-67d46c0d9389") : secret "dns-default-metrics-tls" not found Jul 14 01:00:14.160449 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.160198 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-8cpht"] Jul 14 01:00:14.167532 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:00:14.167489 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83eeb05b_795c_487c_9714_6c1deba18873.slice/crio-48c3def729965b2e4fbcd3acf9a6ac2d3fba85bdee882c568fe12c447d742e5f WatchSource:0}: Error finding container 48c3def729965b2e4fbcd3acf9a6ac2d3fba85bdee882c568fe12c447d742e5f: Status 404 returned error can't find the container with id 48c3def729965b2e4fbcd3acf9a6ac2d3fba85bdee882c568fe12c447d742e5f Jul 14 01:00:14.220275 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.220231 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4"] Jul 14 01:00:14.226137 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:00:14.224883 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c7799c6_a631_4555_b405_cbcf44b861ce.slice/crio-0f152436ce4b4711601a2c6acfcaba516bf771e584239ef8ab54ab07b8f48132 WatchSource:0}: Error finding container 0f152436ce4b4711601a2c6acfcaba516bf771e584239ef8ab54ab07b8f48132: Status 404 returned error can't find the container with id 0f152436ce4b4711601a2c6acfcaba516bf771e584239ef8ab54ab07b8f48132 Jul 14 01:00:14.285693 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.285281 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-tvhvj"] Jul 14 01:00:14.290820 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:00:14.289819 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe11544a_5834_4744_a9d6_a6bd32372ee6.slice/crio-47dc0fd04649d71bd38a712542f133be03c936115324b2a63d004ba2430c0fbc WatchSource:0}: Error finding container 47dc0fd04649d71bd38a712542f133be03c936115324b2a63d004ba2430c0fbc: Status 404 returned error can't find the container with id 47dc0fd04649d71bd38a712542f133be03c936115324b2a63d004ba2430c0fbc Jul 14 01:00:14.307529 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.307454 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7jgwz"] Jul 14 01:00:14.312186 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:00:14.311956 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod791300b3_0b23_48c5_84a8_3b9b52954625.slice/crio-566d0c8a41a7040c6d8d76a51848da29b0a2a3185018a10f0a672a5f3ad3df96 WatchSource:0}: Error finding container 566d0c8a41a7040c6d8d76a51848da29b0a2a3185018a10f0a672a5f3ad3df96: Status 404 returned error can't find the container with id 566d0c8a41a7040c6d8d76a51848da29b0a2a3185018a10f0a672a5f3ad3df96 Jul 14 01:00:14.342230 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.342196 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm"] Jul 14 01:00:14.346064 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:00:14.346034 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d054d1_dce4_479b_9adc_04f7b4feb9f9.slice/crio-6fd114cd7f64e30bb13ec5822316e280b300a0534190d265b0ac05e2c84da4d5 WatchSource:0}: Error finding container 6fd114cd7f64e30bb13ec5822316e280b300a0534190d265b0ac05e2c84da4d5: Status 404 returned error can't find the container with id 6fd114cd7f64e30bb13ec5822316e280b300a0534190d265b0ac05e2c84da4d5 Jul 14 01:00:14.358437 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.358398 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-wgj6h"] Jul 14 01:00:14.361767 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:00:14.361744 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47f717ea_28d1_474f_8c21_1742d49d4306.slice/crio-05ff2d32a1f3396dd75b44c1d0f85d6c11865dc17e264f4bc1757e5ec9289c82 WatchSource:0}: Error finding container 05ff2d32a1f3396dd75b44c1d0f85d6c11865dc17e264f4bc1757e5ec9289c82: Status 404 returned error can't find the container with id 05ff2d32a1f3396dd75b44c1d0f85d6c11865dc17e264f4bc1757e5ec9289c82 Jul 14 01:00:14.558441 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.558353 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-np8n4\" (UID: \"ccb9e582-0ff4-40af-8c5b-38ffe27dea5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" Jul 14 01:00:14.558594 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:14.558449 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:14.558594 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:14.558513 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 01:00:14.558594 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:14.558579 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls podName:ccb9e582-0ff4-40af-8c5b-38ffe27dea5b nodeName:}" failed. No retries permitted until 2026-07-14 01:00:15.558562152 +0000 UTC m=+44.461417633 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls") pod "cluster-samples-operator-65db858894-np8n4" (UID: "ccb9e582-0ff4-40af-8c5b-38ffe27dea5b") : secret "samples-operator-tls" not found Jul 14 01:00:14.558752 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:14.558611 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 01:00:14.558752 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:14.558635 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-646d9864b5-pt6rn: secret "image-registry-tls" not found Jul 14 01:00:14.558752 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:14.558698 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls podName:d1310c00-61a2-4b6a-b23c-7740b8c8bdf0 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:15.558679665 +0000 UTC m=+44.461535164 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls") pod "image-registry-646d9864b5-pt6rn" (UID: "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0") : secret "image-registry-tls" not found Jul 14 01:00:15.025362 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:15.025304 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" event={"ID":"be11544a-5834-4744-a9d6-a6bd32372ee6","Type":"ContainerStarted","Data":"47dc0fd04649d71bd38a712542f133be03c936115324b2a63d004ba2430c0fbc"} Jul 14 01:00:15.026404 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:15.026383 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8cpht" event={"ID":"83eeb05b-795c-487c-9714-6c1deba18873","Type":"ContainerStarted","Data":"48c3def729965b2e4fbcd3acf9a6ac2d3fba85bdee882c568fe12c447d742e5f"} Jul 14 01:00:15.027403 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:15.027384 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-wgj6h" event={"ID":"47f717ea-28d1-474f-8c21-1742d49d4306","Type":"ContainerStarted","Data":"05ff2d32a1f3396dd75b44c1d0f85d6c11865dc17e264f4bc1757e5ec9289c82"} Jul 14 01:00:15.028382 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:15.028362 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" event={"ID":"16d054d1-dce4-479b-9adc-04f7b4feb9f9","Type":"ContainerStarted","Data":"6fd114cd7f64e30bb13ec5822316e280b300a0534190d265b0ac05e2c84da4d5"} Jul 14 01:00:15.029402 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:15.029383 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7jgwz" event={"ID":"791300b3-0b23-48c5-84a8-3b9b52954625","Type":"ContainerStarted","Data":"566d0c8a41a7040c6d8d76a51848da29b0a2a3185018a10f0a672a5f3ad3df96"} Jul 14 01:00:15.030281 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:15.030264 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" event={"ID":"9c7799c6-a631-4555-b405-cbcf44b861ce","Type":"ContainerStarted","Data":"0f152436ce4b4711601a2c6acfcaba516bf771e584239ef8ab54ab07b8f48132"} Jul 14 01:00:15.568426 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:15.568394 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:15.568596 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:15.568486 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-np8n4\" (UID: \"ccb9e582-0ff4-40af-8c5b-38ffe27dea5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" Jul 14 01:00:15.568667 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:15.568648 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 01:00:15.568730 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:15.568712 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls podName:ccb9e582-0ff4-40af-8c5b-38ffe27dea5b nodeName:}" failed. No retries permitted until 2026-07-14 01:00:17.568697635 +0000 UTC m=+46.471553125 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls") pod "cluster-samples-operator-65db858894-np8n4" (UID: "ccb9e582-0ff4-40af-8c5b-38ffe27dea5b") : secret "samples-operator-tls" not found Jul 14 01:00:15.569080 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:15.569064 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 01:00:15.569135 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:15.569082 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-646d9864b5-pt6rn: secret "image-registry-tls" not found Jul 14 01:00:15.569135 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:15.569117 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls podName:d1310c00-61a2-4b6a-b23c-7740b8c8bdf0 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:17.569105068 +0000 UTC m=+46.471960559 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls") pod "image-registry-646d9864b5-pt6rn" (UID: "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0") : secret "image-registry-tls" not found Jul 14 01:00:17.039799 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:17.039756 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7jgwz" event={"ID":"791300b3-0b23-48c5-84a8-3b9b52954625","Type":"ContainerStarted","Data":"1b07b210d4200d75cacb6183dd8d2fb57c52527dc82c130e3dc7b50d0f2fe28c"} Jul 14 01:00:17.587127 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:17.587078 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:17.587345 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:17.587186 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-np8n4\" (UID: \"ccb9e582-0ff4-40af-8c5b-38ffe27dea5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" Jul 14 01:00:17.587345 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:17.587298 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 01:00:17.587345 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:17.587342 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-646d9864b5-pt6rn: secret "image-registry-tls" not found Jul 14 01:00:17.587500 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:17.587361 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 01:00:17.587500 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:17.587414 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls podName:d1310c00-61a2-4b6a-b23c-7740b8c8bdf0 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:21.587391466 +0000 UTC m=+50.490246962 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls") pod "image-registry-646d9864b5-pt6rn" (UID: "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0") : secret "image-registry-tls" not found Jul 14 01:00:17.587500 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:17.587434 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls podName:ccb9e582-0ff4-40af-8c5b-38ffe27dea5b nodeName:}" failed. No retries permitted until 2026-07-14 01:00:21.587424096 +0000 UTC m=+50.490279580 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls") pod "cluster-samples-operator-65db858894-np8n4" (UID: "ccb9e582-0ff4-40af-8c5b-38ffe27dea5b") : secret "samples-operator-tls" not found Jul 14 01:00:21.623519 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:21.623466 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-np8n4\" (UID: \"ccb9e582-0ff4-40af-8c5b-38ffe27dea5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" Jul 14 01:00:21.624112 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:21.623573 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:21.624112 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:21.623649 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 01:00:21.624112 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:21.623705 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 01:00:21.624112 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:21.623719 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-646d9864b5-pt6rn: secret "image-registry-tls" not found Jul 14 01:00:21.624112 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:21.623734 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls podName:ccb9e582-0ff4-40af-8c5b-38ffe27dea5b nodeName:}" failed. No retries permitted until 2026-07-14 01:00:29.623711842 +0000 UTC m=+58.526567333 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls") pod "cluster-samples-operator-65db858894-np8n4" (UID: "ccb9e582-0ff4-40af-8c5b-38ffe27dea5b") : secret "samples-operator-tls" not found Jul 14 01:00:21.624112 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:21.623769 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls podName:d1310c00-61a2-4b6a-b23c-7740b8c8bdf0 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:29.623752121 +0000 UTC m=+58.526607613 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls") pod "image-registry-646d9864b5-pt6rn" (UID: "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0") : secret "image-registry-tls" not found Jul 14 01:00:22.055289 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:22.055196 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" event={"ID":"9c7799c6-a631-4555-b405-cbcf44b861ce","Type":"ContainerStarted","Data":"d54b95a51ece22891e78cc053f5f867f7b6051cf96372f2865236b3ee41e5e44"} Jul 14 01:00:22.056929 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:22.056901 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" event={"ID":"be11544a-5834-4744-a9d6-a6bd32372ee6","Type":"ContainerStarted","Data":"eea90957ecebcef9697e6bddf1b11a506416607e892407f4065b9887e48c5b67"} Jul 14 01:00:22.058611 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:22.058582 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8cpht" event={"ID":"83eeb05b-795c-487c-9714-6c1deba18873","Type":"ContainerStarted","Data":"3b34cf5005d2572db1a3604c225f6dc77104e49bb23ebd111c805d68d89b723b"} Jul 14 01:00:22.060763 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:22.060742 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/0.log" Jul 14 01:00:22.060887 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:22.060778 2582 generic.go:356] "Generic (PLEG): container finished" podID="47f717ea-28d1-474f-8c21-1742d49d4306" containerID="f955bb23ea509e939e756920325c2808cf4521d79ed7a41ddc0a2c84c9938624" exitCode=255 Jul 14 01:00:22.060887 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:22.060842 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-wgj6h" event={"ID":"47f717ea-28d1-474f-8c21-1742d49d4306","Type":"ContainerDied","Data":"f955bb23ea509e939e756920325c2808cf4521d79ed7a41ddc0a2c84c9938624"} Jul 14 01:00:22.061118 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:22.061096 2582 scope.go:117] "RemoveContainer" containerID="f955bb23ea509e939e756920325c2808cf4521d79ed7a41ddc0a2c84c9938624" Jul 14 01:00:22.062861 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:22.062839 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" event={"ID":"16d054d1-dce4-479b-9adc-04f7b4feb9f9","Type":"ContainerStarted","Data":"6053d7b78b4dd7d28282409ba91e2999e310a64d6a30f9b09de87e286cce0448"} Jul 14 01:00:22.070301 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:22.070223 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7jgwz" podStartSLOduration=7.0009854 podStartE2EDuration="9.070194906s" totalImagesPullingTime="2.069209506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:17.055418104 +0000 UTC m=+45.958273605" watchObservedRunningTime="2026-07-14 01:00:22.070194906 +0000 UTC m=+50.973050404" Jul 14 01:00:22.070979 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:22.070846 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" podStartSLOduration=2.026605287 podStartE2EDuration="9.070830816s" totalImagesPullingTime="7.044225529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:22.07054259 +0000 UTC m=+50.973398089" watchObservedRunningTime="2026-07-14 01:00:22.070830816 +0000 UTC m=+50.973686313" Jul 14 01:00:22.086799 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:22.085800 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8cpht" podStartSLOduration=1.984807593 podStartE2EDuration="9.085780751s" totalImagesPullingTime="7.100973158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:22.085465603 +0000 UTC m=+50.988321106" watchObservedRunningTime="2026-07-14 01:00:22.085780751 +0000 UTC m=+50.988636244" Jul 14 01:00:22.103820 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:22.103764 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" podStartSLOduration=2.180130638 podStartE2EDuration="9.103739955s" totalImagesPullingTime="6.923609317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:22.103193055 +0000 UTC m=+51.006048547" watchObservedRunningTime="2026-07-14 01:00:22.103739955 +0000 UTC m=+51.006595455" Jul 14 01:00:22.120749 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:22.120139 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" podStartSLOduration=2.1409185 podStartE2EDuration="9.120118716s" totalImagesPullingTime="6.979200216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:22.119841414 +0000 UTC m=+51.022696913" watchObservedRunningTime="2026-07-14 01:00:22.120118716 +0000 UTC m=+51.022974219" Jul 14 01:00:22.228823 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:22.228787 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:22.229067 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:22.229042 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 01:00:22.229272 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:22.229257 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls podName:a2342051-9f5f-4acc-bfd9-67d46c0d9389 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:38.229233174 +0000 UTC m=+67.132088670 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls") pod "dns-default-jcbh7" (UID: "a2342051-9f5f-4acc-bfd9-67d46c0d9389") : secret "dns-default-metrics-tls" not found Jul 14 01:00:23.067156 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.067130 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:00:23.067643 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.067570 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/0.log" Jul 14 01:00:23.067643 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.067606 2582 generic.go:356] "Generic (PLEG): container finished" podID="47f717ea-28d1-474f-8c21-1742d49d4306" containerID="cbc2e4e29dcc73e78675d6690ac33b8785f2fee724bad961ea4866d4996d0092" exitCode=255 Jul 14 01:00:23.067743 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.067704 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-wgj6h" event={"ID":"47f717ea-28d1-474f-8c21-1742d49d4306","Type":"ContainerDied","Data":"cbc2e4e29dcc73e78675d6690ac33b8785f2fee724bad961ea4866d4996d0092"} Jul 14 01:00:23.067791 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.067757 2582 scope.go:117] "RemoveContainer" containerID="f955bb23ea509e939e756920325c2808cf4521d79ed7a41ddc0a2c84c9938624" Jul 14 01:00:23.067959 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.067928 2582 scope.go:117] "RemoveContainer" containerID="cbc2e4e29dcc73e78675d6690ac33b8785f2fee724bad961ea4866d4996d0092" Jul 14 01:00:23.068147 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:23.068125 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-wgj6h_openshift-console-operator(47f717ea-28d1-474f-8c21-1742d49d4306)\"" pod="openshift-console-operator/console-operator-7866497494-wgj6h" podUID="47f717ea-28d1-474f-8c21-1742d49d4306" Jul 14 01:00:23.305903 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.305865 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-28m79"] Jul 14 01:00:23.347441 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.347409 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-28m79"] Jul 14 01:00:23.347603 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.347536 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:23.349979 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.349957 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 01:00:23.350126 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.350007 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zmdxs\"" Jul 14 01:00:23.350126 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.349958 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 01:00:23.440653 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.440616 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:23.440835 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.440683 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-crio-socket\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:23.440835 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.440723 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-data-volume\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:23.440835 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.440748 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppsch\" (UniqueName: \"kubernetes.io/projected/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-api-access-ppsch\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:23.440835 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.440802 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:23.541795 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.541756 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-data-volume\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:23.541795 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.541796 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppsch\" (UniqueName: \"kubernetes.io/projected/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-api-access-ppsch\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:23.542021 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.541819 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:23.542021 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.541877 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:23.542245 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.541917 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-crio-socket\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:23.542457 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.542411 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-crio-socket\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:23.542578 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:23.542384 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 01:00:23.542725 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.542630 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-data-volume\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:23.542782 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:23.542742 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm podName:f5c67b4e-18dd-4f01-9459-4421a0a4cc64 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:24.042715444 +0000 UTC m=+52.945570943 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm") pod "insights-runtime-extractor-28m79" (UID: "f5c67b4e-18dd-4f01-9459-4421a0a4cc64") : configmap "kube-rbac-proxy" not found Jul 14 01:00:23.542782 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:23.542410 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 01:00:23.543344 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:23.543309 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls podName:f5c67b4e-18dd-4f01-9459-4421a0a4cc64 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:24.043285813 +0000 UTC m=+52.946141486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls") pod "insights-runtime-extractor-28m79" (UID: "f5c67b4e-18dd-4f01-9459-4421a0a4cc64") : secret "insights-runtime-extractor-tls" not found Jul 14 01:00:23.551554 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:23.551522 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppsch\" (UniqueName: \"kubernetes.io/projected/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-api-access-ppsch\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:24.047706 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:24.047673 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:24.047956 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:24.047743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:24.047956 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:24.047820 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 01:00:24.047956 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:24.047837 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 01:00:24.047956 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:24.047902 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm podName:f5c67b4e-18dd-4f01-9459-4421a0a4cc64 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:25.047880353 +0000 UTC m=+53.950735830 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm") pod "insights-runtime-extractor-28m79" (UID: "f5c67b4e-18dd-4f01-9459-4421a0a4cc64") : configmap "kube-rbac-proxy" not found Jul 14 01:00:24.047956 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:24.047920 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls podName:f5c67b4e-18dd-4f01-9459-4421a0a4cc64 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:25.047912298 +0000 UTC m=+53.950767776 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls") pod "insights-runtime-extractor-28m79" (UID: "f5c67b4e-18dd-4f01-9459-4421a0a4cc64") : secret "insights-runtime-extractor-tls" not found Jul 14 01:00:24.072482 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:24.072455 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:00:24.072830 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:24.072813 2582 scope.go:117] "RemoveContainer" containerID="cbc2e4e29dcc73e78675d6690ac33b8785f2fee724bad961ea4866d4996d0092" Jul 14 01:00:24.073008 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:24.072989 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-wgj6h_openshift-console-operator(47f717ea-28d1-474f-8c21-1742d49d4306)\"" pod="openshift-console-operator/console-operator-7866497494-wgj6h" podUID="47f717ea-28d1-474f-8c21-1742d49d4306" Jul 14 01:00:24.152610 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:24.152578 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:24.152766 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:24.152635 2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:25.056736 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.056687 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:25.056930 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.056774 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:25.056930 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:25.056842 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 01:00:25.056930 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:25.056843 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 01:00:25.056930 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:25.056906 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm podName:f5c67b4e-18dd-4f01-9459-4421a0a4cc64 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:27.056892017 +0000 UTC m=+55.959747495 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm") pod "insights-runtime-extractor-28m79" (UID: "f5c67b4e-18dd-4f01-9459-4421a0a4cc64") : configmap "kube-rbac-proxy" not found Jul 14 01:00:25.056930 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:25.056920 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls podName:f5c67b4e-18dd-4f01-9459-4421a0a4cc64 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:27.05691379 +0000 UTC m=+55.959769267 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls") pod "insights-runtime-extractor-28m79" (UID: "f5c67b4e-18dd-4f01-9459-4421a0a4cc64") : secret "insights-runtime-extractor-tls" not found Jul 14 01:00:25.076588 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.076559 2582 scope.go:117] "RemoveContainer" containerID="cbc2e4e29dcc73e78675d6690ac33b8785f2fee724bad961ea4866d4996d0092" Jul 14 01:00:25.076942 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:25.076711 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-wgj6h_openshift-console-operator(47f717ea-28d1-474f-8c21-1742d49d4306)\"" pod="openshift-console-operator/console-operator-7866497494-wgj6h" podUID="47f717ea-28d1-474f-8c21-1742d49d4306" Jul 14 01:00:25.489223 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.489193 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5hpkv_c31e51b7-7bee-4ce8-9517-a1e6814a2641/dns-node-resolver/0.log" Jul 14 01:00:25.585476 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.585442 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-4nf9f"] Jul 14 01:00:25.607732 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.607695 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-4nf9f"] Jul 14 01:00:25.607875 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.607789 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-4nf9f" Jul 14 01:00:25.610259 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.610235 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 14 01:00:25.610402 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.610386 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 14 01:00:25.611370 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.611347 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 14 01:00:25.611475 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.611379 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-zd2zc\"" Jul 14 01:00:25.611475 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.611402 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 14 01:00:25.662675 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.662634 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/faa83b44-988c-44f8-90f9-885e02732668-signing-cabundle\") pod \"service-ca-79f68b9f4f-4nf9f\" (UID: \"faa83b44-988c-44f8-90f9-885e02732668\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4nf9f" Jul 14 01:00:25.662825 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.662687 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzfxf\" (UniqueName: \"kubernetes.io/projected/faa83b44-988c-44f8-90f9-885e02732668-kube-api-access-gzfxf\") pod \"service-ca-79f68b9f4f-4nf9f\" (UID: \"faa83b44-988c-44f8-90f9-885e02732668\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4nf9f" Jul 14 01:00:25.662825 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.662712 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/faa83b44-988c-44f8-90f9-885e02732668-signing-key\") pod \"service-ca-79f68b9f4f-4nf9f\" (UID: \"faa83b44-988c-44f8-90f9-885e02732668\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4nf9f" Jul 14 01:00:25.764094 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.764008 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/faa83b44-988c-44f8-90f9-885e02732668-signing-cabundle\") pod \"service-ca-79f68b9f4f-4nf9f\" (UID: \"faa83b44-988c-44f8-90f9-885e02732668\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4nf9f" Jul 14 01:00:25.764094 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.764051 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzfxf\" (UniqueName: \"kubernetes.io/projected/faa83b44-988c-44f8-90f9-885e02732668-kube-api-access-gzfxf\") pod \"service-ca-79f68b9f4f-4nf9f\" (UID: \"faa83b44-988c-44f8-90f9-885e02732668\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4nf9f" Jul 14 01:00:25.764094 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.764072 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/faa83b44-988c-44f8-90f9-885e02732668-signing-key\") pod \"service-ca-79f68b9f4f-4nf9f\" (UID: \"faa83b44-988c-44f8-90f9-885e02732668\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4nf9f" Jul 14 01:00:25.764801 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.764778 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/faa83b44-988c-44f8-90f9-885e02732668-signing-cabundle\") pod \"service-ca-79f68b9f4f-4nf9f\" (UID: \"faa83b44-988c-44f8-90f9-885e02732668\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4nf9f" Jul 14 01:00:25.766647 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.766625 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/faa83b44-988c-44f8-90f9-885e02732668-signing-key\") pod \"service-ca-79f68b9f4f-4nf9f\" (UID: \"faa83b44-988c-44f8-90f9-885e02732668\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4nf9f" Jul 14 01:00:25.772496 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.772473 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzfxf\" (UniqueName: \"kubernetes.io/projected/faa83b44-988c-44f8-90f9-885e02732668-kube-api-access-gzfxf\") pod \"service-ca-79f68b9f4f-4nf9f\" (UID: \"faa83b44-988c-44f8-90f9-885e02732668\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4nf9f" Jul 14 01:00:25.916498 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:25.916462 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-4nf9f" Jul 14 01:00:26.064415 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:26.064382 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-4nf9f"] Jul 14 01:00:26.068363 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:00:26.068315 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa83b44_988c_44f8_90f9_885e02732668.slice/crio-620bbe9cd28034f4d67568205d4d9db4f69d900d4e9d94ebe961e5bdb1aaae41 WatchSource:0}: Error finding container 620bbe9cd28034f4d67568205d4d9db4f69d900d4e9d94ebe961e5bdb1aaae41: Status 404 returned error can't find the container with id 620bbe9cd28034f4d67568205d4d9db4f69d900d4e9d94ebe961e5bdb1aaae41 Jul 14 01:00:26.080457 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:26.080422 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-4nf9f" event={"ID":"faa83b44-988c-44f8-90f9-885e02732668","Type":"ContainerStarted","Data":"620bbe9cd28034f4d67568205d4d9db4f69d900d4e9d94ebe961e5bdb1aaae41"} Jul 14 01:00:26.490217 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:26.490190 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-24x4q_24004bce-6659-4fc1-a3e5-8a8d85103612/node-ca/0.log" Jul 14 01:00:27.074816 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:27.074770 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:27.075030 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:27.074852 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:27.075030 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:27.074927 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 01:00:27.075030 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:27.075004 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls podName:f5c67b4e-18dd-4f01-9459-4421a0a4cc64 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:31.074987309 +0000 UTC m=+59.977842788 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls") pod "insights-runtime-extractor-28m79" (UID: "f5c67b4e-18dd-4f01-9459-4421a0a4cc64") : secret "insights-runtime-extractor-tls" not found Jul 14 01:00:27.075030 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:27.074941 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 01:00:27.075242 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:27.075075 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm podName:f5c67b4e-18dd-4f01-9459-4421a0a4cc64 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:31.075062441 +0000 UTC m=+59.977917919 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm") pod "insights-runtime-extractor-28m79" (UID: "f5c67b4e-18dd-4f01-9459-4421a0a4cc64") : configmap "kube-rbac-proxy" not found Jul 14 01:00:27.084934 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:27.084901 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-4nf9f" event={"ID":"faa83b44-988c-44f8-90f9-885e02732668","Type":"ContainerStarted","Data":"21dcabd8ed4dcff6df5f239889930f8af42da854ad7284690b15d316a1c4be1a"} Jul 14 01:00:27.099926 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:27.099879 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-4nf9f" podStartSLOduration=2.099864214 podStartE2EDuration="2.099864214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:27.099170707 +0000 UTC m=+56.002026207" watchObservedRunningTime="2026-07-14 01:00:27.099864214 +0000 UTC m=+56.002719749" Jul 14 01:00:27.489906 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:27.489868 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6x4hg_1191145f-7591-45da-9a24-7c20fc700c59/serve-healthcheck-canary/0.log" Jul 14 01:00:28.090953 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:28.090921 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-74cl4_9c7799c6-a631-4555-b405-cbcf44b861ce/kube-storage-version-migrator-operator/0.log" Jul 14 01:00:29.698074 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:29.698032 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:29.698559 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:29.698124 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-np8n4\" (UID: \"ccb9e582-0ff4-40af-8c5b-38ffe27dea5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" Jul 14 01:00:29.698559 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:29.698185 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 01:00:29.698559 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:29.698202 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-646d9864b5-pt6rn: secret "image-registry-tls" not found Jul 14 01:00:29.698559 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:29.698258 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls podName:d1310c00-61a2-4b6a-b23c-7740b8c8bdf0 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:45.69824072 +0000 UTC m=+74.601096232 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls") pod "image-registry-646d9864b5-pt6rn" (UID: "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0") : secret "image-registry-tls" not found Jul 14 01:00:29.700646 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:29.700627 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ccb9e582-0ff4-40af-8c5b-38ffe27dea5b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-np8n4\" (UID: \"ccb9e582-0ff4-40af-8c5b-38ffe27dea5b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" Jul 14 01:00:29.737182 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:29.737148 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" Jul 14 01:00:29.877259 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:29.877228 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4"] Jul 14 01:00:30.095917 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:30.095877 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" event={"ID":"ccb9e582-0ff4-40af-8c5b-38ffe27dea5b","Type":"ContainerStarted","Data":"4bb942e09de225bc256635bdc531bf0d752c558fd2beb16c480b3c8b67c2ea99"} Jul 14 01:00:31.112208 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:31.112176 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:31.112694 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:31.112264 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:31.112694 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:31.112356 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 01:00:31.112694 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:31.112368 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 01:00:31.112694 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:31.112422 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm podName:f5c67b4e-18dd-4f01-9459-4421a0a4cc64 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:39.112402287 +0000 UTC m=+68.015257765 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm") pod "insights-runtime-extractor-28m79" (UID: "f5c67b4e-18dd-4f01-9459-4421a0a4cc64") : configmap "kube-rbac-proxy" not found Jul 14 01:00:31.112694 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:00:31.112545 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls podName:f5c67b4e-18dd-4f01-9459-4421a0a4cc64 nodeName:}" failed. No retries permitted until 2026-07-14 01:00:39.112527139 +0000 UTC m=+68.015382622 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls") pod "insights-runtime-extractor-28m79" (UID: "f5c67b4e-18dd-4f01-9459-4421a0a4cc64") : secret "insights-runtime-extractor-tls" not found Jul 14 01:00:32.994605 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:32.994576 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pqg6h" Jul 14 01:00:33.108334 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:33.108292 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" event={"ID":"ccb9e582-0ff4-40af-8c5b-38ffe27dea5b","Type":"ContainerStarted","Data":"b1ac28c83f79ef85532e8a2b48f697c88244676973bfabaf6ded4ba238e10efb"} Jul 14 01:00:33.108505 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:33.108377 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" event={"ID":"ccb9e582-0ff4-40af-8c5b-38ffe27dea5b","Type":"ContainerStarted","Data":"5d188086033eb5c130b390752cb9f33e35ca3fbaedd0ccdfa3aba5a603253514"} Jul 14 01:00:33.123990 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:33.123931 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-np8n4" podStartSLOduration=17.647605748 podStartE2EDuration="20.123910035s" totalImagesPullingTime="2.476304287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:33.123902239 +0000 UTC m=+62.026757737" watchObservedRunningTime="2026-07-14 01:00:33.123910035 +0000 UTC m=+62.026765535" Jul 14 01:00:36.455829 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.455786 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs\") pod \"network-metrics-daemon-kch6v\" (UID: \"3c724b42-2885-4687-970d-9ef08ddf10ff\") " pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 01:00:36.456297 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.455868 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 01:00:36.458667 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.458646 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 01:00:36.458740 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.458645 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 01:00:36.469369 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.469311 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7953918-10f5-4a00-8b0a-d89348f25314-original-pull-secret\") pod \"global-pull-secret-syncer-rmnsp\" (UID: \"f7953918-10f5-4a00-8b0a-d89348f25314\") " pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 01:00:36.469503 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.469413 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c724b42-2885-4687-970d-9ef08ddf10ff-metrics-certs\") pod \"network-metrics-daemon-kch6v\" (UID: \"3c724b42-2885-4687-970d-9ef08ddf10ff\") " pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 01:00:36.556717 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.556684 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7nq6\" (UniqueName: \"kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6\") pod \"network-check-target-rcg77\" (UID: \"84c558a6-a95a-44c3-8500-3e5d85a08a0a\") " pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 01:00:36.559578 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.559544 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7nq6\" (UniqueName: \"kubernetes.io/projected/84c558a6-a95a-44c3-8500-3e5d85a08a0a-kube-api-access-m7nq6\") pod \"network-check-target-rcg77\" (UID: \"84c558a6-a95a-44c3-8500-3e5d85a08a0a\") " pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 01:00:36.701469 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.701435 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-btv6j\"" Jul 14 01:00:36.706657 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.706605 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rmnsp" Jul 14 01:00:36.709183 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.709162 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 01:00:36.714398 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.714346 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hm62n\"" Jul 14 01:00:36.722173 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.722150 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kch6v" Jul 14 01:00:36.858396 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.858360 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rmnsp"] Jul 14 01:00:36.862455 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:00:36.862422 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7953918_10f5_4a00_8b0a_d89348f25314.slice/crio-8da81a27452c7f1a244da5af1970142e7f5f6e55eb7b2a07487e37af70802e3e WatchSource:0}: Error finding container 8da81a27452c7f1a244da5af1970142e7f5f6e55eb7b2a07487e37af70802e3e: Status 404 returned error can't find the container with id 8da81a27452c7f1a244da5af1970142e7f5f6e55eb7b2a07487e37af70802e3e Jul 14 01:00:36.882416 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.882306 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rcg77"] Jul 14 01:00:36.885057 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:00:36.885032 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84c558a6_a95a_44c3_8500_3e5d85a08a0a.slice/crio-f05e54c8d8ae9ffe3b38b6f73b543f50498717614bd020daadff0a060b18f4b7 WatchSource:0}: Error finding container f05e54c8d8ae9ffe3b38b6f73b543f50498717614bd020daadff0a060b18f4b7: Status 404 returned error can't find the container with id f05e54c8d8ae9ffe3b38b6f73b543f50498717614bd020daadff0a060b18f4b7 Jul 14 01:00:36.913286 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:36.913258 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kch6v"] Jul 14 01:00:37.125552 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:37.125517 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rmnsp" event={"ID":"f7953918-10f5-4a00-8b0a-d89348f25314","Type":"ContainerStarted","Data":"8da81a27452c7f1a244da5af1970142e7f5f6e55eb7b2a07487e37af70802e3e"} Jul 14 01:00:37.126608 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:37.126584 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kch6v" event={"ID":"3c724b42-2885-4687-970d-9ef08ddf10ff","Type":"ContainerStarted","Data":"cc273ff459518224d43fa500b6f0f498d81d773b0b6345fb60c479ab000f732d"} Jul 14 01:00:37.127952 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:37.127933 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rcg77" event={"ID":"84c558a6-a95a-44c3-8500-3e5d85a08a0a","Type":"ContainerStarted","Data":"f57f82e743b2e15cf29fbbaa6ffa0c7fd5fd802eda784a53cbaa7c4bb1dbd62f"} Jul 14 01:00:37.128010 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:37.127970 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rcg77" event={"ID":"84c558a6-a95a-44c3-8500-3e5d85a08a0a","Type":"ContainerStarted","Data":"f05e54c8d8ae9ffe3b38b6f73b543f50498717614bd020daadff0a060b18f4b7"} Jul 14 01:00:37.128072 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:37.128055 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 01:00:37.145790 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:37.145738 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rcg77" podStartSLOduration=66.145721409 podStartE2EDuration="1m6.145721409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:59:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:37.144538489 +0000 UTC m=+66.047393988" watchObservedRunningTime="2026-07-14 01:00:37.145721409 +0000 UTC m=+66.048576886" Jul 14 01:00:38.272794 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:38.272757 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:38.275482 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:38.275452 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2342051-9f5f-4acc-bfd9-67d46c0d9389-metrics-tls\") pod \"dns-default-jcbh7\" (UID: \"a2342051-9f5f-4acc-bfd9-67d46c0d9389\") " pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:38.536024 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:38.535994 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x8vr9\"" Jul 14 01:00:38.544022 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:38.543988 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:38.674902 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:38.674867 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jcbh7"] Jul 14 01:00:38.678129 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:00:38.678102 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2342051_9f5f_4acc_bfd9_67d46c0d9389.slice/crio-90d3d4a131ee12af0e70293c61aef330e10e9f8848526b7058a8d005d7b38b65 WatchSource:0}: Error finding container 90d3d4a131ee12af0e70293c61aef330e10e9f8848526b7058a8d005d7b38b65: Status 404 returned error can't find the container with id 90d3d4a131ee12af0e70293c61aef330e10e9f8848526b7058a8d005d7b38b65 Jul 14 01:00:39.137604 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:39.137563 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kch6v" event={"ID":"3c724b42-2885-4687-970d-9ef08ddf10ff","Type":"ContainerStarted","Data":"06760a063850ce9fcaad30e5e29e26800ff287e26d710d2def738713fbb900b0"} Jul 14 01:00:39.137793 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:39.137640 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kch6v" event={"ID":"3c724b42-2885-4687-970d-9ef08ddf10ff","Type":"ContainerStarted","Data":"d94cea153788b2a639687d3ba617dd657b1946cfe343584e7944528e0d2d5b4d"} Jul 14 01:00:39.138566 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:39.138536 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jcbh7" event={"ID":"a2342051-9f5f-4acc-bfd9-67d46c0d9389","Type":"ContainerStarted","Data":"90d3d4a131ee12af0e70293c61aef330e10e9f8848526b7058a8d005d7b38b65"} Jul 14 01:00:39.153089 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:39.153035 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kch6v" podStartSLOduration=67.019847709 podStartE2EDuration="1m8.15301597s" totalImagesPullingTime="1.133168261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:39.151823684 +0000 UTC m=+68.054679183" watchObservedRunningTime="2026-07-14 01:00:39.15301597 +0000 UTC m=+68.055871469" Jul 14 01:00:39.181429 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:39.181391 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:39.181589 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:39.181492 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:39.181882 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:39.181861 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:39.183907 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:39.183886 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5c67b4e-18dd-4f01-9459-4421a0a4cc64-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-28m79\" (UID: \"f5c67b4e-18dd-4f01-9459-4421a0a4cc64\") " pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:39.258965 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:39.258930 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zmdxs\"" Jul 14 01:00:39.266373 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:39.266345 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-28m79" Jul 14 01:00:39.421564 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:39.421292 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-28m79"] Jul 14 01:00:39.426737 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:00:39.426693 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5c67b4e_18dd_4f01_9459_4421a0a4cc64.slice/crio-d42e48d3dc33c06f66e32b3961edab5cb7b8a00e47159c04ba53327240176cd3 WatchSource:0}: Error finding container d42e48d3dc33c06f66e32b3961edab5cb7b8a00e47159c04ba53327240176cd3: Status 404 returned error can't find the container with id d42e48d3dc33c06f66e32b3961edab5cb7b8a00e47159c04ba53327240176cd3 Jul 14 01:00:39.687085 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:39.687013 2582 scope.go:117] "RemoveContainer" containerID="cbc2e4e29dcc73e78675d6690ac33b8785f2fee724bad961ea4866d4996d0092" Jul 14 01:00:40.144597 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:40.144549 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-28m79" event={"ID":"f5c67b4e-18dd-4f01-9459-4421a0a4cc64","Type":"ContainerStarted","Data":"d42e48d3dc33c06f66e32b3961edab5cb7b8a00e47159c04ba53327240176cd3"} Jul 14 01:00:40.146818 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:40.146777 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:00:40.147078 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:40.147011 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-wgj6h" event={"ID":"47f717ea-28d1-474f-8c21-1742d49d4306","Type":"ContainerStarted","Data":"1dc52476ed0ababf97d34735c951174f2ec08a00939eef549dd801ed2963d20a"} Jul 14 01:00:40.147795 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:40.147743 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:40.833390 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:40.833307 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-wgj6h" Jul 14 01:00:40.850232 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:40.850174 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-wgj6h" podStartSLOduration=20.935789243 podStartE2EDuration="27.850150735s" totalImagesPullingTime="6.914361492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:40.167041825 +0000 UTC m=+69.069897336" watchObservedRunningTime="2026-07-14 01:00:40.850150735 +0000 UTC m=+69.753006234" Jul 14 01:00:45.167367 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:45.167244 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jcbh7" event={"ID":"a2342051-9f5f-4acc-bfd9-67d46c0d9389","Type":"ContainerStarted","Data":"534250d9c33eef622083404a24059604d9c4806559ef3bd798975cee9c537d3b"} Jul 14 01:00:45.167367 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:45.167297 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jcbh7" event={"ID":"a2342051-9f5f-4acc-bfd9-67d46c0d9389","Type":"ContainerStarted","Data":"40d508bc4c95348cafb170bf9f09683f98113f7ed6dc6adf27082e416ece7c7b"} Jul 14 01:00:45.167858 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:45.167463 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:45.168914 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:45.168889 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-28m79" event={"ID":"f5c67b4e-18dd-4f01-9459-4421a0a4cc64","Type":"ContainerStarted","Data":"c7e0345b1d938b32cffb6c9486b8b923d0ead8b1ee390750bdb32854c37d94bf"} Jul 14 01:00:45.170243 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:45.170219 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rmnsp" event={"ID":"f7953918-10f5-4a00-8b0a-d89348f25314","Type":"ContainerStarted","Data":"7363146bdecf36105f94de46a1b5077922ad823d8e80f4913b5e67991f737118"} Jul 14 01:00:45.183460 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:45.183418 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jcbh7" podStartSLOduration=33.010478388 podStartE2EDuration="39.183403041s" totalImagesPullingTime="6.172924653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:45.182809045 +0000 UTC m=+74.085664542" watchObservedRunningTime="2026-07-14 01:00:45.183403041 +0000 UTC m=+74.086258540" Jul 14 01:00:45.196310 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:45.196269 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rmnsp" podStartSLOduration=66.207400789 podStartE2EDuration="1m14.196255075s" totalImagesPullingTime="7.988854286s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:45.195463838 +0000 UTC m=+74.098319331" watchObservedRunningTime="2026-07-14 01:00:45.196255075 +0000 UTC m=+74.099110573" Jul 14 01:00:45.738220 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:45.738192 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:45.740639 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:45.740619 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls\") pod \"image-registry-646d9864b5-pt6rn\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:45.986769 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:45.986691 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gn7kr\"" Jul 14 01:00:45.994860 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:45.994841 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:46.142254 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:46.142219 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-646d9864b5-pt6rn"] Jul 14 01:00:46.145109 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:00:46.145075 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1310c00_61a2_4b6a_b23c_7740b8c8bdf0.slice/crio-8e36bd5c9647fb81e36ae4b7f0c1fa5f4a559e7c099dec053a56a1c22b09efce WatchSource:0}: Error finding container 8e36bd5c9647fb81e36ae4b7f0c1fa5f4a559e7c099dec053a56a1c22b09efce: Status 404 returned error can't find the container with id 8e36bd5c9647fb81e36ae4b7f0c1fa5f4a559e7c099dec053a56a1c22b09efce Jul 14 01:00:46.175549 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:46.175513 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-28m79" event={"ID":"f5c67b4e-18dd-4f01-9459-4421a0a4cc64","Type":"ContainerStarted","Data":"47738928160aa891d36982968f63ea73ccb42c5997f423d077d7dbe5bc6583af"} Jul 14 01:00:46.176609 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:46.176581 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" event={"ID":"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0","Type":"ContainerStarted","Data":"8e36bd5c9647fb81e36ae4b7f0c1fa5f4a559e7c099dec053a56a1c22b09efce"} Jul 14 01:00:47.184592 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.184550 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" event={"ID":"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0","Type":"ContainerStarted","Data":"52527d906f61e5f4d306a9855efcefd4b41b8ce73cb323e07c5bd8c4672179b6"} Jul 14 01:00:47.185067 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.184719 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:00:47.234224 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.234155 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" podStartSLOduration=34.23413422 podStartE2EDuration="34.23413422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:47.232533468 +0000 UTC m=+76.135388967" watchObservedRunningTime="2026-07-14 01:00:47.23413422 +0000 UTC m=+76.136989724" Jul 14 01:00:47.416875 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.416847 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss"] Jul 14 01:00:47.421637 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.421608 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.424057 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.424030 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 14 01:00:47.424602 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.424573 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 14 01:00:47.424715 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.424635 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 14 01:00:47.424783 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.424711 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 14 01:00:47.424837 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.424575 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 14 01:00:47.425120 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.425100 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 14 01:00:47.425289 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.425271 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 14 01:00:47.433657 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.433635 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss"] Jul 14 01:00:47.476095 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.476015 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-646d9864b5-pt6rn"] Jul 14 01:00:47.552200 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.552167 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/69857e1d-e038-41a1-b27d-22b05d10057b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.552440 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.552217 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/69857e1d-e038-41a1-b27d-22b05d10057b-hub\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.552440 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.552316 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/69857e1d-e038-41a1-b27d-22b05d10057b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.552440 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.552376 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xk24\" (UniqueName: \"kubernetes.io/projected/69857e1d-e038-41a1-b27d-22b05d10057b-kube-api-access-5xk24\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.552440 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.552412 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/69857e1d-e038-41a1-b27d-22b05d10057b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.552440 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.552434 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/69857e1d-e038-41a1-b27d-22b05d10057b-ca\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.559077 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.559051 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5885768549-tk4qh"] Jul 14 01:00:47.562515 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.562479 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.574843 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.574813 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5885768549-tk4qh"] Jul 14 01:00:47.652877 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.652843 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xk24\" (UniqueName: \"kubernetes.io/projected/69857e1d-e038-41a1-b27d-22b05d10057b-kube-api-access-5xk24\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.653053 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.652887 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/69857e1d-e038-41a1-b27d-22b05d10057b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.653053 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.652913 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d51c30b-b732-4644-8097-b9f3f62271a9-registry-certificates\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.653053 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.652939 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/69857e1d-e038-41a1-b27d-22b05d10057b-ca\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.653053 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.652969 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d51c30b-b732-4644-8097-b9f3f62271a9-trusted-ca\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.653053 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.653007 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d51c30b-b732-4644-8097-b9f3f62271a9-image-registry-private-configuration\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.653354 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.653157 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/69857e1d-e038-41a1-b27d-22b05d10057b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.653354 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.653204 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d51c30b-b732-4644-8097-b9f3f62271a9-ca-trust-extracted\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.653354 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.653240 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/69857e1d-e038-41a1-b27d-22b05d10057b-hub\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.653354 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.653281 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d51c30b-b732-4644-8097-b9f3f62271a9-installation-pull-secrets\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.653555 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.653373 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d51c30b-b732-4644-8097-b9f3f62271a9-registry-tls\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.653555 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.653398 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d51c30b-b732-4644-8097-b9f3f62271a9-bound-sa-token\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.653555 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.653425 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-979k7\" (UniqueName: \"kubernetes.io/projected/7d51c30b-b732-4644-8097-b9f3f62271a9-kube-api-access-979k7\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.653555 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.653455 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/69857e1d-e038-41a1-b27d-22b05d10057b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.653764 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.653729 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/69857e1d-e038-41a1-b27d-22b05d10057b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.656207 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.656181 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/69857e1d-e038-41a1-b27d-22b05d10057b-ca\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.656349 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.656269 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/69857e1d-e038-41a1-b27d-22b05d10057b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.656589 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.656568 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/69857e1d-e038-41a1-b27d-22b05d10057b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.656646 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.656601 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/69857e1d-e038-41a1-b27d-22b05d10057b-hub\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.668245 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.668212 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xk24\" (UniqueName: \"kubernetes.io/projected/69857e1d-e038-41a1-b27d-22b05d10057b-kube-api-access-5xk24\") pod \"cluster-proxy-proxy-agent-7bd6b784c-9cfss\" (UID: \"69857e1d-e038-41a1-b27d-22b05d10057b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.740957 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.740872 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" Jul 14 01:00:47.754178 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.754148 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d51c30b-b732-4644-8097-b9f3f62271a9-registry-tls\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.754178 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.754183 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d51c30b-b732-4644-8097-b9f3f62271a9-bound-sa-token\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.754403 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.754201 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-979k7\" (UniqueName: \"kubernetes.io/projected/7d51c30b-b732-4644-8097-b9f3f62271a9-kube-api-access-979k7\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.754458 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.754438 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d51c30b-b732-4644-8097-b9f3f62271a9-registry-certificates\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.754515 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.754477 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d51c30b-b732-4644-8097-b9f3f62271a9-trusted-ca\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.754515 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.754505 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d51c30b-b732-4644-8097-b9f3f62271a9-image-registry-private-configuration\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.754615 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.754543 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d51c30b-b732-4644-8097-b9f3f62271a9-ca-trust-extracted\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.754976 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.754948 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d51c30b-b732-4644-8097-b9f3f62271a9-ca-trust-extracted\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.755100 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.755010 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d51c30b-b732-4644-8097-b9f3f62271a9-installation-pull-secrets\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.755355 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.755313 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d51c30b-b732-4644-8097-b9f3f62271a9-registry-certificates\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.755827 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.755618 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d51c30b-b732-4644-8097-b9f3f62271a9-trusted-ca\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.757433 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.757407 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d51c30b-b732-4644-8097-b9f3f62271a9-registry-tls\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.757690 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.757670 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d51c30b-b732-4644-8097-b9f3f62271a9-image-registry-private-configuration\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.757737 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.757714 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d51c30b-b732-4644-8097-b9f3f62271a9-installation-pull-secrets\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.764092 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.764069 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d51c30b-b732-4644-8097-b9f3f62271a9-bound-sa-token\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.765590 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.765568 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-979k7\" (UniqueName: \"kubernetes.io/projected/7d51c30b-b732-4644-8097-b9f3f62271a9-kube-api-access-979k7\") pod \"image-registry-5885768549-tk4qh\" (UID: \"7d51c30b-b732-4644-8097-b9f3f62271a9\") " pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:47.873636 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:47.873604 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:48.537065 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:48.536974 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5885768549-tk4qh"] Jul 14 01:00:48.554100 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:48.554076 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss"] Jul 14 01:00:48.588303 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:00:48.588242 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d51c30b_b732_4644_8097_b9f3f62271a9.slice/crio-9bdbed917959fe4d204cf6c0eb9f75ecbf5f834044c3ed07cce8709277564e03 WatchSource:0}: Error finding container 9bdbed917959fe4d204cf6c0eb9f75ecbf5f834044c3ed07cce8709277564e03: Status 404 returned error can't find the container with id 9bdbed917959fe4d204cf6c0eb9f75ecbf5f834044c3ed07cce8709277564e03 Jul 14 01:00:48.589028 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:00:48.589003 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69857e1d_e038_41a1_b27d_22b05d10057b.slice/crio-93eda9aaecb9fea88b7edd02ba3f03259ce8f148764030a48db33582d33d8e5c WatchSource:0}: Error finding container 93eda9aaecb9fea88b7edd02ba3f03259ce8f148764030a48db33582d33d8e5c: Status 404 returned error can't find the container with id 93eda9aaecb9fea88b7edd02ba3f03259ce8f148764030a48db33582d33d8e5c Jul 14 01:00:49.194223 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:49.194182 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5885768549-tk4qh" event={"ID":"7d51c30b-b732-4644-8097-b9f3f62271a9","Type":"ContainerStarted","Data":"aa2c859ce8a29283410664c879987b8cc2371b5d7ed9f90c908320cdf57014cf"} Jul 14 01:00:49.194405 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:49.194249 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5885768549-tk4qh" event={"ID":"7d51c30b-b732-4644-8097-b9f3f62271a9","Type":"ContainerStarted","Data":"9bdbed917959fe4d204cf6c0eb9f75ecbf5f834044c3ed07cce8709277564e03"} Jul 14 01:00:49.194405 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:49.194276 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:00:49.196275 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:49.196251 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-28m79" event={"ID":"f5c67b4e-18dd-4f01-9459-4421a0a4cc64","Type":"ContainerStarted","Data":"ea4f7c10bf3dbaaad51da82950a78063c41e98b19fae30d3d5144306e2d800b5"} Jul 14 01:00:49.197447 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:49.197422 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" event={"ID":"69857e1d-e038-41a1-b27d-22b05d10057b","Type":"ContainerStarted","Data":"93eda9aaecb9fea88b7edd02ba3f03259ce8f148764030a48db33582d33d8e5c"} Jul 14 01:00:49.213875 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:49.213819 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5885768549-tk4qh" podStartSLOduration=2.21380441 podStartE2EDuration="2.21380441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:49.212612256 +0000 UTC m=+78.115467756" watchObservedRunningTime="2026-07-14 01:00:49.21380441 +0000 UTC m=+78.116659908" Jul 14 01:00:49.247139 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:49.247082 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-28m79" podStartSLOduration=17.191932971 podStartE2EDuration="26.2470645s" totalImagesPullingTime="9.055131529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:00:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:49.24611272 +0000 UTC m=+78.148968219" watchObservedRunningTime="2026-07-14 01:00:49.2470645 +0000 UTC m=+78.149920007" Jul 14 01:00:55.179259 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:55.179228 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jcbh7" Jul 14 01:00:55.233724 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:55.233687 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" event={"ID":"69857e1d-e038-41a1-b27d-22b05d10057b","Type":"ContainerStarted","Data":"733f9c9499457b6acb0d8018c5d7a3773fccde77e71bd08634adc8fc0fe06200"} Jul 14 01:00:55.233724 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:55.233731 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" event={"ID":"69857e1d-e038-41a1-b27d-22b05d10057b","Type":"ContainerStarted","Data":"b1134158958218fdbe91dcb6a3e8abbb7384e2b7197c00d09ce67504637dc632"} Jul 14 01:00:55.233958 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:55.233744 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" event={"ID":"69857e1d-e038-41a1-b27d-22b05d10057b","Type":"ContainerStarted","Data":"87e7ef1ec8298b2edfafe65e8cad82a3ee07b9cf4cf0268d44ba82971acedcdb"} Jul 14 01:00:55.252272 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:00:55.252220 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bd6b784c-9cfss" podStartSLOduration=2.248948213 podStartE2EDuration="8.252203019s" totalImagesPullingTime="6.003254806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:00:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:00:55.251473385 +0000 UTC m=+84.154328901" watchObservedRunningTime="2026-07-14 01:00:55.252203019 +0000 UTC m=+84.155058518" Jul 14 01:01:05.407257 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.407221 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ln8nk"] Jul 14 01:01:05.412687 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.412657 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.415470 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.415439 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 01:01:05.415623 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.415441 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fhkh5\"" Jul 14 01:01:05.416536 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.416511 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 01:01:05.416696 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.416535 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 01:01:05.416808 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.416705 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 01:01:05.416808 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.416553 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 01:01:05.416977 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.416960 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 01:01:05.507883 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.507843 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99b530e3-52a1-4b82-9c62-f3334a2c8017-metrics-client-ca\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.507883 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.507886 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-tls\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.508149 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.507907 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-accelerators-collector-config\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.508149 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.507971 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/99b530e3-52a1-4b82-9c62-f3334a2c8017-root\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.508149 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.508026 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-wtmp\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.508149 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.508070 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99b530e3-52a1-4b82-9c62-f3334a2c8017-sys\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.508149 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.508096 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-textfile\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.508149 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.508124 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.508469 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.508163 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltk47\" (UniqueName: \"kubernetes.io/projected/99b530e3-52a1-4b82-9c62-f3334a2c8017-kube-api-access-ltk47\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.608796 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.608756 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99b530e3-52a1-4b82-9c62-f3334a2c8017-metrics-client-ca\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.608986 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.608810 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-tls\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.608986 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.608839 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-accelerators-collector-config\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.608986 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.608871 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/99b530e3-52a1-4b82-9c62-f3334a2c8017-root\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.608986 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.608903 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-wtmp\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.608986 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.608943 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99b530e3-52a1-4b82-9c62-f3334a2c8017-sys\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.608986 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.608968 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-textfile\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.609302 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.608999 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/99b530e3-52a1-4b82-9c62-f3334a2c8017-root\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.609302 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:01:05.609019 2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 14 01:01:05.609302 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.609005 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.609302 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:01:05.609095 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-tls podName:99b530e3-52a1-4b82-9c62-f3334a2c8017 nodeName:}" failed. No retries permitted until 2026-07-14 01:01:06.109072576 +0000 UTC m=+95.011928054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-tls") pod "node-exporter-ln8nk" (UID: "99b530e3-52a1-4b82-9c62-f3334a2c8017") : secret "node-exporter-tls" not found Jul 14 01:01:05.609302 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.609117 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltk47\" (UniqueName: \"kubernetes.io/projected/99b530e3-52a1-4b82-9c62-f3334a2c8017-kube-api-access-ltk47\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.609302 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.609118 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99b530e3-52a1-4b82-9c62-f3334a2c8017-sys\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.609302 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.609252 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-wtmp\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.609642 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.609474 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99b530e3-52a1-4b82-9c62-f3334a2c8017-metrics-client-ca\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.609642 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.609618 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-textfile\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.609708 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.609626 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-accelerators-collector-config\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.611759 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.611737 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:05.618486 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:05.618464 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltk47\" (UniqueName: \"kubernetes.io/projected/99b530e3-52a1-4b82-9c62-f3334a2c8017-kube-api-access-ltk47\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:06.114778 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:06.114725 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-tls\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:06.117208 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:06.117176 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/99b530e3-52a1-4b82-9c62-f3334a2c8017-node-exporter-tls\") pod \"node-exporter-ln8nk\" (UID: \"99b530e3-52a1-4b82-9c62-f3334a2c8017\") " pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:06.323727 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:06.323686 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ln8nk" Jul 14 01:01:06.336510 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:01:06.336482 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99b530e3_52a1_4b82_9c62_f3334a2c8017.slice/crio-e4fc049045c9759bd3c8eec838ab67e901dce32f7b657a6a01d66e82a3677e1f WatchSource:0}: Error finding container e4fc049045c9759bd3c8eec838ab67e901dce32f7b657a6a01d66e82a3677e1f: Status 404 returned error can't find the container with id e4fc049045c9759bd3c8eec838ab67e901dce32f7b657a6a01d66e82a3677e1f Jul 14 01:01:07.279627 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:07.279585 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ln8nk" event={"ID":"99b530e3-52a1-4b82-9c62-f3334a2c8017","Type":"ContainerStarted","Data":"e4fc049045c9759bd3c8eec838ab67e901dce32f7b657a6a01d66e82a3677e1f"} Jul 14 01:01:08.135556 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:08.135526 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rcg77" Jul 14 01:01:08.284878 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:08.284838 2582 generic.go:356] "Generic (PLEG): container finished" podID="99b530e3-52a1-4b82-9c62-f3334a2c8017" containerID="dfae3a12d93c26141778044a9e4a2267f047537addd28812a61b9d0d20b2e427" exitCode=0 Jul 14 01:01:08.285266 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:08.284884 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ln8nk" event={"ID":"99b530e3-52a1-4b82-9c62-f3334a2c8017","Type":"ContainerDied","Data":"dfae3a12d93c26141778044a9e4a2267f047537addd28812a61b9d0d20b2e427"} Jul 14 01:01:09.202934 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:09.202903 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:01:09.291659 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:09.291608 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ln8nk" event={"ID":"99b530e3-52a1-4b82-9c62-f3334a2c8017","Type":"ContainerStarted","Data":"cfe3aa472c2f98a9222bd195ec88d43feb97ab6ae52b31dbb43b14e6f49a2a33"} Jul 14 01:01:09.292110 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:09.291694 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ln8nk" event={"ID":"99b530e3-52a1-4b82-9c62-f3334a2c8017","Type":"ContainerStarted","Data":"acb4bbe19b4d7423922a94f224b83ec8b4def8d0f3100253e1914cf245bc1791"} Jul 14 01:01:09.308835 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:09.308774 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ln8nk" podStartSLOduration=3.240957352 podStartE2EDuration="4.308754004s" totalImagesPullingTime="998.088615ms" totalInitContainerRuntime="69.708037ms" isStatefulPod=true podCreationTimestamp="2026-07-14 01:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:01:09.307825629 +0000 UTC m=+98.210681129" watchObservedRunningTime="2026-07-14 01:01:09.308754004 +0000 UTC m=+98.211609503" Jul 14 01:01:10.205436 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:10.205398 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5885768549-tk4qh" Jul 14 01:01:14.217912 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.217870 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" podUID="d1310c00-61a2-4b6a-b23c-7740b8c8bdf0" containerName="registry" containerID="cri-o://52527d906f61e5f4d306a9855efcefd4b41b8ce73cb323e07c5bd8c4672179b6" gracePeriod=30 Jul 14 01:01:14.467642 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.467617 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:01:14.479050 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.478958 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-certificates\") pod \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " Jul 14 01:01:14.479050 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.479016 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-ca-trust-extracted\") pod \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " Jul 14 01:01:14.479050 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.479052 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5cl2\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-kube-api-access-p5cl2\") pod \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " Jul 14 01:01:14.479313 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.479092 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-trusted-ca\") pod \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " Jul 14 01:01:14.479313 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.479118 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls\") pod \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " Jul 14 01:01:14.479313 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.479155 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-image-registry-private-configuration\") pod \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " Jul 14 01:01:14.479313 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.479179 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-installation-pull-secrets\") pod \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " Jul 14 01:01:14.479313 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.479216 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-bound-sa-token\") pod \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\" (UID: \"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0\") " Jul 14 01:01:14.479575 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.479499 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0" (UID: "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:01:14.479634 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.479575 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0" (UID: "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:01:14.482087 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.482047 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0" (UID: "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:01:14.482087 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.482054 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0" (UID: "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 01:01:14.482282 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.482183 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-kube-api-access-p5cl2" (OuterVolumeSpecName: "kube-api-access-p5cl2") pod "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0" (UID: "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0"). InnerVolumeSpecName "kube-api-access-p5cl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:01:14.482282 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.482213 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0" (UID: "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 01:01:14.482282 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.482228 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0" (UID: "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:01:14.490385 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.490347 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0" (UID: "d1310c00-61a2-4b6a-b23c-7740b8c8bdf0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:01:14.580592 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.580550 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-trusted-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:01:14.580592 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.580582 2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-tls\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:01:14.580592 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.580593 2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-image-registry-private-configuration\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:01:14.580811 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.580603 2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-installation-pull-secrets\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:01:14.580811 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.580614 2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-bound-sa-token\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:01:14.580811 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.580622 2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-registry-certificates\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:01:14.580811 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.580630 2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-ca-trust-extracted\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:01:14.580811 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:14.580637 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5cl2\" (UniqueName: \"kubernetes.io/projected/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0-kube-api-access-p5cl2\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:01:15.314859 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:15.314821 2582 generic.go:356] "Generic (PLEG): container finished" podID="d1310c00-61a2-4b6a-b23c-7740b8c8bdf0" containerID="52527d906f61e5f4d306a9855efcefd4b41b8ce73cb323e07c5bd8c4672179b6" exitCode=0 Jul 14 01:01:15.315244 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:15.314872 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" Jul 14 01:01:15.315244 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:15.314907 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" event={"ID":"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0","Type":"ContainerDied","Data":"52527d906f61e5f4d306a9855efcefd4b41b8ce73cb323e07c5bd8c4672179b6"} Jul 14 01:01:15.315244 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:15.314981 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-646d9864b5-pt6rn" event={"ID":"d1310c00-61a2-4b6a-b23c-7740b8c8bdf0","Type":"ContainerDied","Data":"8e36bd5c9647fb81e36ae4b7f0c1fa5f4a559e7c099dec053a56a1c22b09efce"} Jul 14 01:01:15.315244 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:15.315007 2582 scope.go:117] "RemoveContainer" containerID="52527d906f61e5f4d306a9855efcefd4b41b8ce73cb323e07c5bd8c4672179b6" Jul 14 01:01:15.323483 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:15.323463 2582 scope.go:117] "RemoveContainer" containerID="52527d906f61e5f4d306a9855efcefd4b41b8ce73cb323e07c5bd8c4672179b6" Jul 14 01:01:15.323788 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:01:15.323767 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52527d906f61e5f4d306a9855efcefd4b41b8ce73cb323e07c5bd8c4672179b6\": container with ID starting with 52527d906f61e5f4d306a9855efcefd4b41b8ce73cb323e07c5bd8c4672179b6 not found: ID does not exist" containerID="52527d906f61e5f4d306a9855efcefd4b41b8ce73cb323e07c5bd8c4672179b6" Jul 14 01:01:15.323835 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:15.323798 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52527d906f61e5f4d306a9855efcefd4b41b8ce73cb323e07c5bd8c4672179b6"} err="failed to get container status \"52527d906f61e5f4d306a9855efcefd4b41b8ce73cb323e07c5bd8c4672179b6\": rpc error: code = NotFound desc = could not find container \"52527d906f61e5f4d306a9855efcefd4b41b8ce73cb323e07c5bd8c4672179b6\": container with ID starting with 52527d906f61e5f4d306a9855efcefd4b41b8ce73cb323e07c5bd8c4672179b6 not found: ID does not exist" Jul 14 01:01:15.338014 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:15.337982 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-646d9864b5-pt6rn"] Jul 14 01:01:15.341488 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:15.341465 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-646d9864b5-pt6rn"] Jul 14 01:01:15.691337 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:15.691290 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1310c00-61a2-4b6a-b23c-7740b8c8bdf0" path="/var/lib/kubelet/pods/d1310c00-61a2-4b6a-b23c-7740b8c8bdf0/volumes" Jul 14 01:01:27.359604 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:27.359571 2582 generic.go:356] "Generic (PLEG): container finished" podID="9c7799c6-a631-4555-b405-cbcf44b861ce" containerID="d54b95a51ece22891e78cc053f5f867f7b6051cf96372f2865236b3ee41e5e44" exitCode=0 Jul 14 01:01:27.360033 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:27.359627 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" event={"ID":"9c7799c6-a631-4555-b405-cbcf44b861ce","Type":"ContainerDied","Data":"d54b95a51ece22891e78cc053f5f867f7b6051cf96372f2865236b3ee41e5e44"} Jul 14 01:01:27.360033 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:27.359916 2582 scope.go:117] "RemoveContainer" containerID="d54b95a51ece22891e78cc053f5f867f7b6051cf96372f2865236b3ee41e5e44" Jul 14 01:01:28.364608 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:28.364573 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-74cl4" event={"ID":"9c7799c6-a631-4555-b405-cbcf44b861ce","Type":"ContainerStarted","Data":"4a5ddebbed3bc4fde57b46a99eafb3ff1163c762298dc4e73e4006a6a61e195c"} Jul 14 01:01:32.384193 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:32.384154 2582 generic.go:356] "Generic (PLEG): container finished" podID="be11544a-5834-4744-a9d6-a6bd32372ee6" containerID="eea90957ecebcef9697e6bddf1b11a506416607e892407f4065b9887e48c5b67" exitCode=0 Jul 14 01:01:32.384193 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:32.384197 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" event={"ID":"be11544a-5834-4744-a9d6-a6bd32372ee6","Type":"ContainerDied","Data":"eea90957ecebcef9697e6bddf1b11a506416607e892407f4065b9887e48c5b67"} Jul 14 01:01:32.384712 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:32.384530 2582 scope.go:117] "RemoveContainer" containerID="eea90957ecebcef9697e6bddf1b11a506416607e892407f4065b9887e48c5b67" Jul 14 01:01:33.394066 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:33.394027 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-tvhvj" event={"ID":"be11544a-5834-4744-a9d6-a6bd32372ee6","Type":"ContainerStarted","Data":"aea498d8ad39bee85a99a112faf88ef85e004334297e42eb9bf6f109befa29ca"} Jul 14 01:01:42.432749 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:42.432713 2582 generic.go:356] "Generic (PLEG): container finished" podID="16d054d1-dce4-479b-9adc-04f7b4feb9f9" containerID="6053d7b78b4dd7d28282409ba91e2999e310a64d6a30f9b09de87e286cce0448" exitCode=0 Jul 14 01:01:42.433173 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:42.432769 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" event={"ID":"16d054d1-dce4-479b-9adc-04f7b4feb9f9","Type":"ContainerDied","Data":"6053d7b78b4dd7d28282409ba91e2999e310a64d6a30f9b09de87e286cce0448"} Jul 14 01:01:42.433173 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:42.433110 2582 scope.go:117] "RemoveContainer" containerID="6053d7b78b4dd7d28282409ba91e2999e310a64d6a30f9b09de87e286cce0448" Jul 14 01:01:43.438594 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:01:43.438557 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-42xrm" event={"ID":"16d054d1-dce4-479b-9adc-04f7b4feb9f9","Type":"ContainerStarted","Data":"38aee5ca99367630815c989a4df2d1eafd4b2361bd8bdbc49d84c03888a89719"} Jul 14 01:03:24.342050 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.342014 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq"] Jul 14 01:03:24.342528 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.342283 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1310c00-61a2-4b6a-b23c-7740b8c8bdf0" containerName="registry" Jul 14 01:03:24.342528 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.342299 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1310c00-61a2-4b6a-b23c-7740b8c8bdf0" containerName="registry" Jul 14 01:03:24.342528 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.342372 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1310c00-61a2-4b6a-b23c-7740b8c8bdf0" containerName="registry" Jul 14 01:03:24.345238 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.345222 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" Jul 14 01:03:24.347796 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.347771 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 01:03:24.347923 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.347893 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 01:03:24.348861 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.348838 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5d4tv\"" Jul 14 01:03:24.353517 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.353492 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq"] Jul 14 01:03:24.419147 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.419111 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08a57a5a-d89b-4034-90a4-3bccbcef5dba-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq\" (UID: \"08a57a5a-d89b-4034-90a4-3bccbcef5dba\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" Jul 14 01:03:24.419147 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.419148 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08a57a5a-d89b-4034-90a4-3bccbcef5dba-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq\" (UID: \"08a57a5a-d89b-4034-90a4-3bccbcef5dba\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" Jul 14 01:03:24.419400 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.419177 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kdcl\" (UniqueName: \"kubernetes.io/projected/08a57a5a-d89b-4034-90a4-3bccbcef5dba-kube-api-access-2kdcl\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq\" (UID: \"08a57a5a-d89b-4034-90a4-3bccbcef5dba\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" Jul 14 01:03:24.519627 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.519594 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08a57a5a-d89b-4034-90a4-3bccbcef5dba-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq\" (UID: \"08a57a5a-d89b-4034-90a4-3bccbcef5dba\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" Jul 14 01:03:24.519627 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.519631 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08a57a5a-d89b-4034-90a4-3bccbcef5dba-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq\" (UID: \"08a57a5a-d89b-4034-90a4-3bccbcef5dba\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" Jul 14 01:03:24.519902 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.519665 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kdcl\" (UniqueName: \"kubernetes.io/projected/08a57a5a-d89b-4034-90a4-3bccbcef5dba-kube-api-access-2kdcl\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq\" (UID: \"08a57a5a-d89b-4034-90a4-3bccbcef5dba\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" Jul 14 01:03:24.520065 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.520046 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08a57a5a-d89b-4034-90a4-3bccbcef5dba-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq\" (UID: \"08a57a5a-d89b-4034-90a4-3bccbcef5dba\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" Jul 14 01:03:24.520128 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.520080 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08a57a5a-d89b-4034-90a4-3bccbcef5dba-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq\" (UID: \"08a57a5a-d89b-4034-90a4-3bccbcef5dba\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" Jul 14 01:03:24.528316 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.528293 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kdcl\" (UniqueName: \"kubernetes.io/projected/08a57a5a-d89b-4034-90a4-3bccbcef5dba-kube-api-access-2kdcl\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq\" (UID: \"08a57a5a-d89b-4034-90a4-3bccbcef5dba\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" Jul 14 01:03:24.655016 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.654915 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" Jul 14 01:03:24.792817 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.792791 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq"] Jul 14 01:03:24.795421 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:03:24.795376 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a57a5a_d89b_4034_90a4_3bccbcef5dba.slice/crio-9d86f801d88a8a4a37b4d087e5ccb051b496b6ffa2b3fe3bfba8608bd5bfdeff WatchSource:0}: Error finding container 9d86f801d88a8a4a37b4d087e5ccb051b496b6ffa2b3fe3bfba8608bd5bfdeff: Status 404 returned error can't find the container with id 9d86f801d88a8a4a37b4d087e5ccb051b496b6ffa2b3fe3bfba8608bd5bfdeff Jul 14 01:03:24.801705 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:24.801678 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" event={"ID":"08a57a5a-d89b-4034-90a4-3bccbcef5dba","Type":"ContainerStarted","Data":"9d86f801d88a8a4a37b4d087e5ccb051b496b6ffa2b3fe3bfba8608bd5bfdeff"} Jul 14 01:03:32.837280 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:32.837241 2582 generic.go:356] "Generic (PLEG): container finished" podID="08a57a5a-d89b-4034-90a4-3bccbcef5dba" containerID="ad27e8a50e9f66cc9abb94d1e148311defbb508755cd16ffe39cbef46f73a085" exitCode=0 Jul 14 01:03:32.837686 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:32.837304 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" event={"ID":"08a57a5a-d89b-4034-90a4-3bccbcef5dba","Type":"ContainerDied","Data":"ad27e8a50e9f66cc9abb94d1e148311defbb508755cd16ffe39cbef46f73a085"} Jul 14 01:03:36.858431 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:36.858394 2582 generic.go:356] "Generic (PLEG): container finished" podID="08a57a5a-d89b-4034-90a4-3bccbcef5dba" containerID="5cfc232c9be72a20b3e42195d2577f6ac07024b2366e4bcc8c87a64426b1ff59" exitCode=0 Jul 14 01:03:36.858822 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:36.858457 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" event={"ID":"08a57a5a-d89b-4034-90a4-3bccbcef5dba","Type":"ContainerDied","Data":"5cfc232c9be72a20b3e42195d2577f6ac07024b2366e4bcc8c87a64426b1ff59"} Jul 14 01:03:44.889863 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:44.889824 2582 generic.go:356] "Generic (PLEG): container finished" podID="08a57a5a-d89b-4034-90a4-3bccbcef5dba" containerID="3cea9677b10f70b19200be092628d7c9cd42b04c61e7053cb15adab9d3ada62e" exitCode=0 Jul 14 01:03:44.890205 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:44.889888 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" event={"ID":"08a57a5a-d89b-4034-90a4-3bccbcef5dba","Type":"ContainerDied","Data":"3cea9677b10f70b19200be092628d7c9cd42b04c61e7053cb15adab9d3ada62e"} Jul 14 01:03:46.021286 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:46.021261 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" Jul 14 01:03:46.108659 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:46.108615 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08a57a5a-d89b-4034-90a4-3bccbcef5dba-bundle\") pod \"08a57a5a-d89b-4034-90a4-3bccbcef5dba\" (UID: \"08a57a5a-d89b-4034-90a4-3bccbcef5dba\") " Jul 14 01:03:46.108828 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:46.108711 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kdcl\" (UniqueName: \"kubernetes.io/projected/08a57a5a-d89b-4034-90a4-3bccbcef5dba-kube-api-access-2kdcl\") pod \"08a57a5a-d89b-4034-90a4-3bccbcef5dba\" (UID: \"08a57a5a-d89b-4034-90a4-3bccbcef5dba\") " Jul 14 01:03:46.108828 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:46.108747 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08a57a5a-d89b-4034-90a4-3bccbcef5dba-util\") pod \"08a57a5a-d89b-4034-90a4-3bccbcef5dba\" (UID: \"08a57a5a-d89b-4034-90a4-3bccbcef5dba\") " Jul 14 01:03:46.109547 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:46.109512 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08a57a5a-d89b-4034-90a4-3bccbcef5dba-bundle" (OuterVolumeSpecName: "bundle") pod "08a57a5a-d89b-4034-90a4-3bccbcef5dba" (UID: "08a57a5a-d89b-4034-90a4-3bccbcef5dba"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:03:46.111169 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:46.111137 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08a57a5a-d89b-4034-90a4-3bccbcef5dba-kube-api-access-2kdcl" (OuterVolumeSpecName: "kube-api-access-2kdcl") pod "08a57a5a-d89b-4034-90a4-3bccbcef5dba" (UID: "08a57a5a-d89b-4034-90a4-3bccbcef5dba"). InnerVolumeSpecName "kube-api-access-2kdcl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:03:46.114316 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:46.114292 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08a57a5a-d89b-4034-90a4-3bccbcef5dba-util" (OuterVolumeSpecName: "util") pod "08a57a5a-d89b-4034-90a4-3bccbcef5dba" (UID: "08a57a5a-d89b-4034-90a4-3bccbcef5dba"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:03:46.209704 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:46.209623 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08a57a5a-d89b-4034-90a4-3bccbcef5dba-bundle\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:03:46.209704 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:46.209651 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kdcl\" (UniqueName: \"kubernetes.io/projected/08a57a5a-d89b-4034-90a4-3bccbcef5dba-kube-api-access-2kdcl\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:03:46.209704 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:46.209663 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08a57a5a-d89b-4034-90a4-3bccbcef5dba-util\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:03:46.898621 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:46.898581 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" event={"ID":"08a57a5a-d89b-4034-90a4-3bccbcef5dba","Type":"ContainerDied","Data":"9d86f801d88a8a4a37b4d087e5ccb051b496b6ffa2b3fe3bfba8608bd5bfdeff"} Jul 14 01:03:46.898621 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:46.898608 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754077vcq" Jul 14 01:03:46.898621 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:46.898629 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d86f801d88a8a4a37b4d087e5ccb051b496b6ffa2b3fe3bfba8608bd5bfdeff" Jul 14 01:03:52.298255 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.298224 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rhl95"] Jul 14 01:03:52.298705 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.298510 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08a57a5a-d89b-4034-90a4-3bccbcef5dba" containerName="util" Jul 14 01:03:52.298705 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.298522 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="08a57a5a-d89b-4034-90a4-3bccbcef5dba" containerName="util" Jul 14 01:03:52.298705 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.298534 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08a57a5a-d89b-4034-90a4-3bccbcef5dba" containerName="extract" Jul 14 01:03:52.298705 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.298539 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="08a57a5a-d89b-4034-90a4-3bccbcef5dba" containerName="extract" Jul 14 01:03:52.298705 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.298558 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08a57a5a-d89b-4034-90a4-3bccbcef5dba" containerName="pull" Jul 14 01:03:52.298705 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.298565 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="08a57a5a-d89b-4034-90a4-3bccbcef5dba" containerName="pull" Jul 14 01:03:52.298705 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.298608 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="08a57a5a-d89b-4034-90a4-3bccbcef5dba" containerName="extract" Jul 14 01:03:52.301310 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.301292 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rhl95" Jul 14 01:03:52.304097 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.304071 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 14 01:03:52.304203 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.304096 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-cqwjw\"" Jul 14 01:03:52.304253 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.304210 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 14 01:03:52.316759 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.316729 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rhl95"] Jul 14 01:03:52.462843 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.462802 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s625k\" (UniqueName: \"kubernetes.io/projected/064ec811-9c58-41a2-b14f-13c7567e2d09-kube-api-access-s625k\") pod \"cert-manager-operator-controller-manager-55c444bf86-rhl95\" (UID: \"064ec811-9c58-41a2-b14f-13c7567e2d09\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rhl95" Jul 14 01:03:52.463038 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.462949 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/064ec811-9c58-41a2-b14f-13c7567e2d09-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rhl95\" (UID: \"064ec811-9c58-41a2-b14f-13c7567e2d09\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rhl95" Jul 14 01:03:52.564373 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.564230 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/064ec811-9c58-41a2-b14f-13c7567e2d09-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rhl95\" (UID: \"064ec811-9c58-41a2-b14f-13c7567e2d09\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rhl95" Jul 14 01:03:52.564373 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.564278 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s625k\" (UniqueName: \"kubernetes.io/projected/064ec811-9c58-41a2-b14f-13c7567e2d09-kube-api-access-s625k\") pod \"cert-manager-operator-controller-manager-55c444bf86-rhl95\" (UID: \"064ec811-9c58-41a2-b14f-13c7567e2d09\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rhl95" Jul 14 01:03:52.564725 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.564703 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/064ec811-9c58-41a2-b14f-13c7567e2d09-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rhl95\" (UID: \"064ec811-9c58-41a2-b14f-13c7567e2d09\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rhl95" Jul 14 01:03:52.572552 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.572527 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s625k\" (UniqueName: \"kubernetes.io/projected/064ec811-9c58-41a2-b14f-13c7567e2d09-kube-api-access-s625k\") pod \"cert-manager-operator-controller-manager-55c444bf86-rhl95\" (UID: \"064ec811-9c58-41a2-b14f-13c7567e2d09\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rhl95" Jul 14 01:03:52.610439 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.610411 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rhl95" Jul 14 01:03:52.746748 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.746713 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rhl95"] Jul 14 01:03:52.750751 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:03:52.750726 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod064ec811_9c58_41a2_b14f_13c7567e2d09.slice/crio-f5f17cd075827b7e67c06ab3b1467848dc62180d5e9e07cc0f3d4bed4807b9da WatchSource:0}: Error finding container f5f17cd075827b7e67c06ab3b1467848dc62180d5e9e07cc0f3d4bed4807b9da: Status 404 returned error can't find the container with id f5f17cd075827b7e67c06ab3b1467848dc62180d5e9e07cc0f3d4bed4807b9da Jul 14 01:03:52.922258 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:52.922225 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rhl95" event={"ID":"064ec811-9c58-41a2-b14f-13c7567e2d09","Type":"ContainerStarted","Data":"f5f17cd075827b7e67c06ab3b1467848dc62180d5e9e07cc0f3d4bed4807b9da"} Jul 14 01:03:55.937643 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:55.937596 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rhl95" event={"ID":"064ec811-9c58-41a2-b14f-13c7567e2d09","Type":"ContainerStarted","Data":"8db598a3bfe2c35f72781ffdfb595ee606c45e036ed2ec55012b451c4cc69e1f"} Jul 14 01:03:55.959790 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:03:55.959738 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rhl95" podStartSLOduration=1.154552967 podStartE2EDuration="3.959724093s" totalImagesPullingTime="2.805171126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:03:55.958808011 +0000 UTC m=+264.861663736" watchObservedRunningTime="2026-07-14 01:03:55.959724093 +0000 UTC m=+264.862579592" Jul 14 01:04:31.538723 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:04:31.538695 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:04:31.541800 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:04:31.541763 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:04:31.551608 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:04:31.551581 2582 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 01:06:07.405282 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:07.405245 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trs2n"] Jul 14 01:06:07.408276 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:07.408259 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trs2n" Jul 14 01:06:07.410713 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:07.410692 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 14 01:06:07.411828 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:07.411810 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 14 01:06:07.411919 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:07.411866 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-d9rbt\"" Jul 14 01:06:07.416397 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:07.416366 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trs2n"] Jul 14 01:06:07.570092 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:07.570054 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcnq7\" (UniqueName: \"kubernetes.io/projected/b4dde0a0-4861-48c4-9c1a-17260a17b610-kube-api-access-pcnq7\") pod \"openshift-lws-operator-fd8ccff4c-trs2n\" (UID: \"b4dde0a0-4861-48c4-9c1a-17260a17b610\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trs2n" Jul 14 01:06:07.570259 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:07.570143 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4dde0a0-4861-48c4-9c1a-17260a17b610-tmp\") pod \"openshift-lws-operator-fd8ccff4c-trs2n\" (UID: \"b4dde0a0-4861-48c4-9c1a-17260a17b610\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trs2n" Jul 14 01:06:07.671393 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:07.671286 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcnq7\" (UniqueName: \"kubernetes.io/projected/b4dde0a0-4861-48c4-9c1a-17260a17b610-kube-api-access-pcnq7\") pod \"openshift-lws-operator-fd8ccff4c-trs2n\" (UID: \"b4dde0a0-4861-48c4-9c1a-17260a17b610\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trs2n" Jul 14 01:06:07.671393 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:07.671362 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4dde0a0-4861-48c4-9c1a-17260a17b610-tmp\") pod \"openshift-lws-operator-fd8ccff4c-trs2n\" (UID: \"b4dde0a0-4861-48c4-9c1a-17260a17b610\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trs2n" Jul 14 01:06:07.671681 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:07.671666 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4dde0a0-4861-48c4-9c1a-17260a17b610-tmp\") pod \"openshift-lws-operator-fd8ccff4c-trs2n\" (UID: \"b4dde0a0-4861-48c4-9c1a-17260a17b610\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trs2n" Jul 14 01:06:07.680039 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:07.680012 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcnq7\" (UniqueName: \"kubernetes.io/projected/b4dde0a0-4861-48c4-9c1a-17260a17b610-kube-api-access-pcnq7\") pod \"openshift-lws-operator-fd8ccff4c-trs2n\" (UID: \"b4dde0a0-4861-48c4-9c1a-17260a17b610\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trs2n" Jul 14 01:06:07.717264 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:07.717230 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trs2n" Jul 14 01:06:07.850427 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:07.850275 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trs2n"] Jul 14 01:06:07.853141 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:06:07.853110 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4dde0a0_4861_48c4_9c1a_17260a17b610.slice/crio-a84b8488902d677042b89405d9a2fe6fdbda190d5aa4af1756e634811aac0a91 WatchSource:0}: Error finding container a84b8488902d677042b89405d9a2fe6fdbda190d5aa4af1756e634811aac0a91: Status 404 returned error can't find the container with id a84b8488902d677042b89405d9a2fe6fdbda190d5aa4af1756e634811aac0a91 Jul 14 01:06:07.854535 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:07.854515 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 01:06:08.472314 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:08.472276 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trs2n" event={"ID":"b4dde0a0-4861-48c4-9c1a-17260a17b610","Type":"ContainerStarted","Data":"a84b8488902d677042b89405d9a2fe6fdbda190d5aa4af1756e634811aac0a91"} Jul 14 01:06:11.489988 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:11.489950 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trs2n" event={"ID":"b4dde0a0-4861-48c4-9c1a-17260a17b610","Type":"ContainerStarted","Data":"872ccb369e402c07f97ebf33e817a3046a22e50fd9d51737e9daef70d49fb70a"} Jul 14 01:06:11.504946 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:11.504888 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-trs2n" podStartSLOduration=1.526104033 podStartE2EDuration="4.504869486s" totalImagesPullingTime="2.978765453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:06:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:06:11.504212 +0000 UTC m=+400.407067500" watchObservedRunningTime="2026-07-14 01:06:11.504869486 +0000 UTC m=+400.407724986" Jul 14 01:06:13.701679 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:13.701645 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj"] Jul 14 01:06:13.705081 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:13.705065 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" Jul 14 01:06:13.707516 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:13.707493 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 01:06:13.707631 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:13.707503 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 01:06:13.708369 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:13.708356 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5d4tv\"" Jul 14 01:06:13.712423 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:13.712388 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj"] Jul 14 01:06:13.818210 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:13.818173 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcr67\" (UniqueName: \"kubernetes.io/projected/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-kube-api-access-vcr67\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj\" (UID: \"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" Jul 14 01:06:13.818210 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:13.818210 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj\" (UID: \"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" Jul 14 01:06:13.818464 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:13.818250 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj\" (UID: \"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" Jul 14 01:06:13.919380 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:13.919340 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj\" (UID: \"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" Jul 14 01:06:13.919567 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:13.919420 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcr67\" (UniqueName: \"kubernetes.io/projected/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-kube-api-access-vcr67\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj\" (UID: \"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" Jul 14 01:06:13.919567 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:13.919442 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj\" (UID: \"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" Jul 14 01:06:13.919715 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:13.919693 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj\" (UID: \"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" Jul 14 01:06:13.919768 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:13.919730 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj\" (UID: \"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" Jul 14 01:06:13.928564 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:13.928538 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcr67\" (UniqueName: \"kubernetes.io/projected/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-kube-api-access-vcr67\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj\" (UID: \"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" Jul 14 01:06:14.015450 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:14.015355 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" Jul 14 01:06:14.154072 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:14.154039 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj"] Jul 14 01:06:14.157365 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:06:14.157317 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod044f3bc4_a67a_422d_a33d_a0a6a1b5e74d.slice/crio-975b55a4afba2ee21a1f63d39b838f0b8132742dec51e3c7ef686e6bb372da60 WatchSource:0}: Error finding container 975b55a4afba2ee21a1f63d39b838f0b8132742dec51e3c7ef686e6bb372da60: Status 404 returned error can't find the container with id 975b55a4afba2ee21a1f63d39b838f0b8132742dec51e3c7ef686e6bb372da60 Jul 14 01:06:14.507856 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:14.507819 2582 generic.go:356] "Generic (PLEG): container finished" podID="044f3bc4-a67a-422d-a33d-a0a6a1b5e74d" containerID="e95b9c909f03b18ee6db84256556b8e002b4a9ad755c950b155ca82f6e4e2f2b" exitCode=0 Jul 14 01:06:14.508029 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:14.507910 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" event={"ID":"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d","Type":"ContainerDied","Data":"e95b9c909f03b18ee6db84256556b8e002b4a9ad755c950b155ca82f6e4e2f2b"} Jul 14 01:06:14.508029 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:14.507993 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" event={"ID":"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d","Type":"ContainerStarted","Data":"975b55a4afba2ee21a1f63d39b838f0b8132742dec51e3c7ef686e6bb372da60"} Jul 14 01:06:15.514306 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:15.514269 2582 generic.go:356] "Generic (PLEG): container finished" podID="044f3bc4-a67a-422d-a33d-a0a6a1b5e74d" containerID="53094caf252ffc1a56dfa1f90510c2e8d9680c79bcde748b67a2c4339b0758d8" exitCode=0 Jul 14 01:06:15.514692 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:15.514366 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" event={"ID":"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d","Type":"ContainerDied","Data":"53094caf252ffc1a56dfa1f90510c2e8d9680c79bcde748b67a2c4339b0758d8"} Jul 14 01:06:16.521012 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:16.520972 2582 generic.go:356] "Generic (PLEG): container finished" podID="044f3bc4-a67a-422d-a33d-a0a6a1b5e74d" containerID="38aee1619ea225333d3905f3d8cca80555077d8609b9ca85646b4a5da03ac3e1" exitCode=0 Jul 14 01:06:16.521404 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:16.521064 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" event={"ID":"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d","Type":"ContainerDied","Data":"38aee1619ea225333d3905f3d8cca80555077d8609b9ca85646b4a5da03ac3e1"} Jul 14 01:06:17.654291 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:17.654268 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" Jul 14 01:06:17.744383 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:17.744272 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-bundle\") pod \"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d\" (UID: \"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d\") " Jul 14 01:06:17.744383 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:17.744343 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcr67\" (UniqueName: \"kubernetes.io/projected/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-kube-api-access-vcr67\") pod \"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d\" (UID: \"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d\") " Jul 14 01:06:17.744383 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:17.744375 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-util\") pod \"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d\" (UID: \"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d\") " Jul 14 01:06:17.745057 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:17.745029 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-bundle" (OuterVolumeSpecName: "bundle") pod "044f3bc4-a67a-422d-a33d-a0a6a1b5e74d" (UID: "044f3bc4-a67a-422d-a33d-a0a6a1b5e74d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:06:17.746590 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:17.746565 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-kube-api-access-vcr67" (OuterVolumeSpecName: "kube-api-access-vcr67") pod "044f3bc4-a67a-422d-a33d-a0a6a1b5e74d" (UID: "044f3bc4-a67a-422d-a33d-a0a6a1b5e74d"). InnerVolumeSpecName "kube-api-access-vcr67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:06:17.750538 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:17.750458 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-util" (OuterVolumeSpecName: "util") pod "044f3bc4-a67a-422d-a33d-a0a6a1b5e74d" (UID: "044f3bc4-a67a-422d-a33d-a0a6a1b5e74d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:06:17.845182 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:17.845149 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-util\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:06:17.845182 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:17.845177 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-bundle\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:06:17.845182 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:17.845188 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcr67\" (UniqueName: \"kubernetes.io/projected/044f3bc4-a67a-422d-a33d-a0a6a1b5e74d-kube-api-access-vcr67\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:06:18.531131 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:18.531097 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" event={"ID":"044f3bc4-a67a-422d-a33d-a0a6a1b5e74d","Type":"ContainerDied","Data":"975b55a4afba2ee21a1f63d39b838f0b8132742dec51e3c7ef686e6bb372da60"} Jul 14 01:06:18.531295 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:18.531144 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="975b55a4afba2ee21a1f63d39b838f0b8132742dec51e3c7ef686e6bb372da60" Jul 14 01:06:18.531295 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:18.531141 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmhqj" Jul 14 01:06:28.847833 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.847794 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v"] Jul 14 01:06:28.848210 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.848099 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="044f3bc4-a67a-422d-a33d-a0a6a1b5e74d" containerName="pull" Jul 14 01:06:28.848210 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.848110 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="044f3bc4-a67a-422d-a33d-a0a6a1b5e74d" containerName="pull" Jul 14 01:06:28.848210 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.848119 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="044f3bc4-a67a-422d-a33d-a0a6a1b5e74d" containerName="util" Jul 14 01:06:28.848210 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.848124 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="044f3bc4-a67a-422d-a33d-a0a6a1b5e74d" containerName="util" Jul 14 01:06:28.848210 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.848133 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="044f3bc4-a67a-422d-a33d-a0a6a1b5e74d" containerName="extract" Jul 14 01:06:28.848210 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.848138 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="044f3bc4-a67a-422d-a33d-a0a6a1b5e74d" containerName="extract" Jul 14 01:06:28.848210 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.848188 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="044f3bc4-a67a-422d-a33d-a0a6a1b5e74d" containerName="extract" Jul 14 01:06:28.851153 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.851134 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:28.853967 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.853932 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 01:06:28.854062 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.854024 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pscwn\"" Jul 14 01:06:28.854062 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.854041 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 01:06:28.854428 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.854396 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 01:06:28.854537 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.854462 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 01:06:28.865262 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.865236 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v"] Jul 14 01:06:28.928604 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.928562 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d400929-90e2-4141-820a-1770487931c6-webhook-cert\") pod \"opendatahub-operator-controller-manager-554b6949c8-8k86v\" (UID: \"8d400929-90e2-4141-820a-1770487931c6\") " pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:28.928813 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.928620 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8d400929-90e2-4141-820a-1770487931c6-manifests\") pod \"opendatahub-operator-controller-manager-554b6949c8-8k86v\" (UID: \"8d400929-90e2-4141-820a-1770487931c6\") " pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:28.928813 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.928688 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d400929-90e2-4141-820a-1770487931c6-tmp\") pod \"opendatahub-operator-controller-manager-554b6949c8-8k86v\" (UID: \"8d400929-90e2-4141-820a-1770487931c6\") " pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:28.928813 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.928724 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gstjh\" (UniqueName: \"kubernetes.io/projected/8d400929-90e2-4141-820a-1770487931c6-kube-api-access-gstjh\") pod \"opendatahub-operator-controller-manager-554b6949c8-8k86v\" (UID: \"8d400929-90e2-4141-820a-1770487931c6\") " pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:28.928813 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:28.928778 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8d400929-90e2-4141-820a-1770487931c6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-554b6949c8-8k86v\" (UID: \"8d400929-90e2-4141-820a-1770487931c6\") " pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:29.029353 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:29.029280 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d400929-90e2-4141-820a-1770487931c6-tmp\") pod \"opendatahub-operator-controller-manager-554b6949c8-8k86v\" (UID: \"8d400929-90e2-4141-820a-1770487931c6\") " pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:29.029552 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:29.029365 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gstjh\" (UniqueName: \"kubernetes.io/projected/8d400929-90e2-4141-820a-1770487931c6-kube-api-access-gstjh\") pod \"opendatahub-operator-controller-manager-554b6949c8-8k86v\" (UID: \"8d400929-90e2-4141-820a-1770487931c6\") " pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:29.029552 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:29.029414 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8d400929-90e2-4141-820a-1770487931c6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-554b6949c8-8k86v\" (UID: \"8d400929-90e2-4141-820a-1770487931c6\") " pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:29.029552 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:29.029468 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d400929-90e2-4141-820a-1770487931c6-webhook-cert\") pod \"opendatahub-operator-controller-manager-554b6949c8-8k86v\" (UID: \"8d400929-90e2-4141-820a-1770487931c6\") " pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:29.029552 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:29.029494 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8d400929-90e2-4141-820a-1770487931c6-manifests\") pod \"opendatahub-operator-controller-manager-554b6949c8-8k86v\" (UID: \"8d400929-90e2-4141-820a-1770487931c6\") " pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:29.029906 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:29.029885 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8d400929-90e2-4141-820a-1770487931c6-manifests\") pod \"opendatahub-operator-controller-manager-554b6949c8-8k86v\" (UID: \"8d400929-90e2-4141-820a-1770487931c6\") " pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:29.031726 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:29.031703 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d400929-90e2-4141-820a-1770487931c6-tmp\") pod \"opendatahub-operator-controller-manager-554b6949c8-8k86v\" (UID: \"8d400929-90e2-4141-820a-1770487931c6\") " pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:29.032123 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:29.032102 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8d400929-90e2-4141-820a-1770487931c6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-554b6949c8-8k86v\" (UID: \"8d400929-90e2-4141-820a-1770487931c6\") " pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:29.032162 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:29.032102 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d400929-90e2-4141-820a-1770487931c6-webhook-cert\") pod \"opendatahub-operator-controller-manager-554b6949c8-8k86v\" (UID: \"8d400929-90e2-4141-820a-1770487931c6\") " pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:29.037341 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:29.037305 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gstjh\" (UniqueName: \"kubernetes.io/projected/8d400929-90e2-4141-820a-1770487931c6-kube-api-access-gstjh\") pod \"opendatahub-operator-controller-manager-554b6949c8-8k86v\" (UID: \"8d400929-90e2-4141-820a-1770487931c6\") " pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:29.161549 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:29.161447 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:29.302433 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:29.302236 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v"] Jul 14 01:06:29.305509 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:06:29.305476 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d400929_90e2_4141_820a_1770487931c6.slice/crio-77d1222822d0388f4d06bdbe8a84c0d4476568697d9144c23fe0ccf0578f52b2 WatchSource:0}: Error finding container 77d1222822d0388f4d06bdbe8a84c0d4476568697d9144c23fe0ccf0578f52b2: Status 404 returned error can't find the container with id 77d1222822d0388f4d06bdbe8a84c0d4476568697d9144c23fe0ccf0578f52b2 Jul 14 01:06:29.580227 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:29.580146 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" event={"ID":"8d400929-90e2-4141-820a-1770487931c6","Type":"ContainerStarted","Data":"77d1222822d0388f4d06bdbe8a84c0d4476568697d9144c23fe0ccf0578f52b2"} Jul 14 01:06:33.600921 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:33.600876 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" event={"ID":"8d400929-90e2-4141-820a-1770487931c6","Type":"ContainerStarted","Data":"b88c22e0850541576b74f72b60e0465bf11719b31272f356211338e391445820"} Jul 14 01:06:34.606422 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:34.606384 2582 generic.go:356] "Generic (PLEG): container finished" podID="8d400929-90e2-4141-820a-1770487931c6" containerID="b88c22e0850541576b74f72b60e0465bf11719b31272f356211338e391445820" exitCode=0 Jul 14 01:06:34.606862 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:34.606465 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" event={"ID":"8d400929-90e2-4141-820a-1770487931c6","Type":"ContainerDied","Data":"b88c22e0850541576b74f72b60e0465bf11719b31272f356211338e391445820"} Jul 14 01:06:35.612728 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:35.612692 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" event={"ID":"8d400929-90e2-4141-820a-1770487931c6","Type":"ContainerStarted","Data":"ed1220aa1526b01459293ecf3abbb60e356273191ea75a194ba4f3d96443c045"} Jul 14 01:06:35.613123 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:35.612839 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:35.633611 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:35.633556 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" podStartSLOduration=2.798668209 podStartE2EDuration="7.633537217s" totalImagesPullingTime="3.788055455s" totalInitContainerRuntime="1.046813553s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:06:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:06:35.631888698 +0000 UTC m=+424.534744197" watchObservedRunningTime="2026-07-14 01:06:35.633537217 +0000 UTC m=+424.536392716" Jul 14 01:06:46.619405 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:46.619371 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-554b6949c8-8k86v" Jul 14 01:06:48.891096 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:48.891061 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rp96x"] Jul 14 01:06:48.907388 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:48.907358 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rp96x"] Jul 14 01:06:48.907558 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:48.907432 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-rp96x" Jul 14 01:06:48.909908 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:48.909882 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 14 01:06:48.910055 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:48.909960 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-65qlh\"" Jul 14 01:06:48.993963 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:48.993926 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69sf9\" (UniqueName: \"kubernetes.io/projected/00ba49f8-ce8b-42dc-8396-57c324cbdace-kube-api-access-69sf9\") pod \"odh-model-controller-7998c67dc4-rp96x\" (UID: \"00ba49f8-ce8b-42dc-8396-57c324cbdace\") " pod="opendatahub/odh-model-controller-7998c67dc4-rp96x" Jul 14 01:06:48.994119 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:48.993969 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00ba49f8-ce8b-42dc-8396-57c324cbdace-cert\") pod \"odh-model-controller-7998c67dc4-rp96x\" (UID: \"00ba49f8-ce8b-42dc-8396-57c324cbdace\") " pod="opendatahub/odh-model-controller-7998c67dc4-rp96x" Jul 14 01:06:49.094851 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.094806 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69sf9\" (UniqueName: \"kubernetes.io/projected/00ba49f8-ce8b-42dc-8396-57c324cbdace-kube-api-access-69sf9\") pod \"odh-model-controller-7998c67dc4-rp96x\" (UID: \"00ba49f8-ce8b-42dc-8396-57c324cbdace\") " pod="opendatahub/odh-model-controller-7998c67dc4-rp96x" Jul 14 01:06:49.095019 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.094858 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00ba49f8-ce8b-42dc-8396-57c324cbdace-cert\") pod \"odh-model-controller-7998c67dc4-rp96x\" (UID: \"00ba49f8-ce8b-42dc-8396-57c324cbdace\") " pod="opendatahub/odh-model-controller-7998c67dc4-rp96x" Jul 14 01:06:49.097498 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.097473 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00ba49f8-ce8b-42dc-8396-57c324cbdace-cert\") pod \"odh-model-controller-7998c67dc4-rp96x\" (UID: \"00ba49f8-ce8b-42dc-8396-57c324cbdace\") " pod="opendatahub/odh-model-controller-7998c67dc4-rp96x" Jul 14 01:06:49.103719 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.103698 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69sf9\" (UniqueName: \"kubernetes.io/projected/00ba49f8-ce8b-42dc-8396-57c324cbdace-kube-api-access-69sf9\") pod \"odh-model-controller-7998c67dc4-rp96x\" (UID: \"00ba49f8-ce8b-42dc-8396-57c324cbdace\") " pod="opendatahub/odh-model-controller-7998c67dc4-rp96x" Jul 14 01:06:49.218696 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.218602 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-rp96x" Jul 14 01:06:49.351879 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.351750 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rp96x"] Jul 14 01:06:49.354575 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:06:49.354551 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00ba49f8_ce8b_42dc_8396_57c324cbdace.slice/crio-943b130ba50223b3f9608b1355a52d832ab8451d4fc9ebb85e9b09cd5eed5d65 WatchSource:0}: Error finding container 943b130ba50223b3f9608b1355a52d832ab8451d4fc9ebb85e9b09cd5eed5d65: Status 404 returned error can't find the container with id 943b130ba50223b3f9608b1355a52d832ab8451d4fc9ebb85e9b09cd5eed5d65 Jul 14 01:06:49.523698 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.523619 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-422x7"] Jul 14 01:06:49.528073 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.528053 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-422x7" Jul 14 01:06:49.530905 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.530883 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 14 01:06:49.531040 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.530908 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-fbgf2\"" Jul 14 01:06:49.534005 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.533982 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-422x7"] Jul 14 01:06:49.599482 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.599448 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb0445d6-8136-4677-a929-dfd78207f3b4-tls-certs\") pod \"model-serving-api-54d89bd5fc-422x7\" (UID: \"eb0445d6-8136-4677-a929-dfd78207f3b4\") " pod="opendatahub/model-serving-api-54d89bd5fc-422x7" Jul 14 01:06:49.599674 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.599562 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klxrk\" (UniqueName: \"kubernetes.io/projected/eb0445d6-8136-4677-a929-dfd78207f3b4-kube-api-access-klxrk\") pod \"model-serving-api-54d89bd5fc-422x7\" (UID: \"eb0445d6-8136-4677-a929-dfd78207f3b4\") " pod="opendatahub/model-serving-api-54d89bd5fc-422x7" Jul 14 01:06:49.673234 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.673194 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rp96x" event={"ID":"00ba49f8-ce8b-42dc-8396-57c324cbdace","Type":"ContainerStarted","Data":"943b130ba50223b3f9608b1355a52d832ab8451d4fc9ebb85e9b09cd5eed5d65"} Jul 14 01:06:49.700850 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.700805 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klxrk\" (UniqueName: \"kubernetes.io/projected/eb0445d6-8136-4677-a929-dfd78207f3b4-kube-api-access-klxrk\") pod \"model-serving-api-54d89bd5fc-422x7\" (UID: \"eb0445d6-8136-4677-a929-dfd78207f3b4\") " pod="opendatahub/model-serving-api-54d89bd5fc-422x7" Jul 14 01:06:49.700988 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.700881 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb0445d6-8136-4677-a929-dfd78207f3b4-tls-certs\") pod \"model-serving-api-54d89bd5fc-422x7\" (UID: \"eb0445d6-8136-4677-a929-dfd78207f3b4\") " pod="opendatahub/model-serving-api-54d89bd5fc-422x7" Jul 14 01:06:49.703363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.703342 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb0445d6-8136-4677-a929-dfd78207f3b4-tls-certs\") pod \"model-serving-api-54d89bd5fc-422x7\" (UID: \"eb0445d6-8136-4677-a929-dfd78207f3b4\") " pod="opendatahub/model-serving-api-54d89bd5fc-422x7" Jul 14 01:06:49.708284 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.708268 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klxrk\" (UniqueName: \"kubernetes.io/projected/eb0445d6-8136-4677-a929-dfd78207f3b4-kube-api-access-klxrk\") pod \"model-serving-api-54d89bd5fc-422x7\" (UID: \"eb0445d6-8136-4677-a929-dfd78207f3b4\") " pod="opendatahub/model-serving-api-54d89bd5fc-422x7" Jul 14 01:06:49.840369 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.840309 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-422x7" Jul 14 01:06:49.968731 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:49.968680 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-422x7"] Jul 14 01:06:49.971304 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:06:49.971277 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb0445d6_8136_4677_a929_dfd78207f3b4.slice/crio-9f172e8330664499be7d2c091979c43c0add6c384dfd787af4fd91e7e30c1232 WatchSource:0}: Error finding container 9f172e8330664499be7d2c091979c43c0add6c384dfd787af4fd91e7e30c1232: Status 404 returned error can't find the container with id 9f172e8330664499be7d2c091979c43c0add6c384dfd787af4fd91e7e30c1232 Jul 14 01:06:50.679439 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:50.679398 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-422x7" event={"ID":"eb0445d6-8136-4677-a929-dfd78207f3b4","Type":"ContainerStarted","Data":"9f172e8330664499be7d2c091979c43c0add6c384dfd787af4fd91e7e30c1232"} Jul 14 01:06:53.693972 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:53.693927 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rp96x" event={"ID":"00ba49f8-ce8b-42dc-8396-57c324cbdace","Type":"ContainerStarted","Data":"f1db25b81ad9726adec4d681e5b682bbfaa151b92f8a6e875f59ab954fb1250a"} Jul 14 01:06:53.694471 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:53.693999 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-rp96x" Jul 14 01:06:53.695511 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:53.695487 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-422x7" event={"ID":"eb0445d6-8136-4677-a929-dfd78207f3b4","Type":"ContainerStarted","Data":"1e5929a7562d4e936077036a8ae3225d8e9501e4347c0bae3da204e3ed531ede"} Jul 14 01:06:53.695600 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:53.695543 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-422x7" Jul 14 01:06:53.709788 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:53.709702 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-rp96x" podStartSLOduration=1.561409629 podStartE2EDuration="5.709684086s" totalImagesPullingTime="4.148274457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:06:53.708471951 +0000 UTC m=+442.611327450" watchObservedRunningTime="2026-07-14 01:06:53.709684086 +0000 UTC m=+442.612539587" Jul 14 01:06:53.723297 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:53.723246 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-422x7" podStartSLOduration=1.240566557 podStartE2EDuration="4.72322875s" totalImagesPullingTime="3.482662193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:06:53.722544884 +0000 UTC m=+442.625400383" watchObservedRunningTime="2026-07-14 01:06:53.72322875 +0000 UTC m=+442.626084249" Jul 14 01:06:54.149157 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.149115 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2"] Jul 14 01:06:54.152813 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.152787 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" Jul 14 01:06:54.155238 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.155200 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 14 01:06:54.155461 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.155441 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 14 01:06:54.155558 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.155456 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 01:06:54.155699 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.155676 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-x48rj\"" Jul 14 01:06:54.156304 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.156281 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 01:06:54.161545 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.161523 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2"] Jul 14 01:06:54.245996 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.245959 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdf94a72-0712-48e2-8131-a13bfcd34872-tls-certs\") pod \"kube-auth-proxy-54d956c5b-x5bv2\" (UID: \"fdf94a72-0712-48e2-8131-a13bfcd34872\") " pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" Jul 14 01:06:54.246155 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.246011 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh6cl\" (UniqueName: \"kubernetes.io/projected/fdf94a72-0712-48e2-8131-a13bfcd34872-kube-api-access-vh6cl\") pod \"kube-auth-proxy-54d956c5b-x5bv2\" (UID: \"fdf94a72-0712-48e2-8131-a13bfcd34872\") " pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" Jul 14 01:06:54.246155 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.246046 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fdf94a72-0712-48e2-8131-a13bfcd34872-tmp\") pod \"kube-auth-proxy-54d956c5b-x5bv2\" (UID: \"fdf94a72-0712-48e2-8131-a13bfcd34872\") " pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" Jul 14 01:06:54.347549 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.347500 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdf94a72-0712-48e2-8131-a13bfcd34872-tls-certs\") pod \"kube-auth-proxy-54d956c5b-x5bv2\" (UID: \"fdf94a72-0712-48e2-8131-a13bfcd34872\") " pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" Jul 14 01:06:54.347723 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.347562 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh6cl\" (UniqueName: \"kubernetes.io/projected/fdf94a72-0712-48e2-8131-a13bfcd34872-kube-api-access-vh6cl\") pod \"kube-auth-proxy-54d956c5b-x5bv2\" (UID: \"fdf94a72-0712-48e2-8131-a13bfcd34872\") " pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" Jul 14 01:06:54.347723 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.347591 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fdf94a72-0712-48e2-8131-a13bfcd34872-tmp\") pod \"kube-auth-proxy-54d956c5b-x5bv2\" (UID: \"fdf94a72-0712-48e2-8131-a13bfcd34872\") " pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" Jul 14 01:06:54.347723 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:06:54.347684 2582 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 14 01:06:54.347836 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:06:54.347766 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdf94a72-0712-48e2-8131-a13bfcd34872-tls-certs podName:fdf94a72-0712-48e2-8131-a13bfcd34872 nodeName:}" failed. No retries permitted until 2026-07-14 01:06:54.847743956 +0000 UTC m=+443.750599445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/fdf94a72-0712-48e2-8131-a13bfcd34872-tls-certs") pod "kube-auth-proxy-54d956c5b-x5bv2" (UID: "fdf94a72-0712-48e2-8131-a13bfcd34872") : secret "kube-auth-proxy-tls" not found Jul 14 01:06:54.350058 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.350038 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fdf94a72-0712-48e2-8131-a13bfcd34872-tmp\") pod \"kube-auth-proxy-54d956c5b-x5bv2\" (UID: \"fdf94a72-0712-48e2-8131-a13bfcd34872\") " pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" Jul 14 01:06:54.355959 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.355935 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh6cl\" (UniqueName: \"kubernetes.io/projected/fdf94a72-0712-48e2-8131-a13bfcd34872-kube-api-access-vh6cl\") pod \"kube-auth-proxy-54d956c5b-x5bv2\" (UID: \"fdf94a72-0712-48e2-8131-a13bfcd34872\") " pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" Jul 14 01:06:54.852109 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:54.852060 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdf94a72-0712-48e2-8131-a13bfcd34872-tls-certs\") pod \"kube-auth-proxy-54d956c5b-x5bv2\" (UID: \"fdf94a72-0712-48e2-8131-a13bfcd34872\") " pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" Jul 14 01:06:54.852622 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:06:54.852220 2582 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 14 01:06:54.852622 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:06:54.852282 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdf94a72-0712-48e2-8131-a13bfcd34872-tls-certs podName:fdf94a72-0712-48e2-8131-a13bfcd34872 nodeName:}" failed. No retries permitted until 2026-07-14 01:06:55.852264341 +0000 UTC m=+444.755119830 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/fdf94a72-0712-48e2-8131-a13bfcd34872-tls-certs") pod "kube-auth-proxy-54d956c5b-x5bv2" (UID: "fdf94a72-0712-48e2-8131-a13bfcd34872") : secret "kube-auth-proxy-tls" not found Jul 14 01:06:55.386119 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.386083 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb"] Jul 14 01:06:55.393207 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.393170 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.395817 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.395794 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 01:06:55.395947 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.395853 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qvvkh\"" Jul 14 01:06:55.395947 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.395860 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 01:06:55.396114 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.396099 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 01:06:55.396156 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.396120 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 01:06:55.403663 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.403643 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb"] Jul 14 01:06:55.456774 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.456733 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9662917b-5681-4a8e-afe3-055f6ebcb23d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.456932 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.456798 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9662917b-5681-4a8e-afe3-055f6ebcb23d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.456932 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.456829 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q77t5\" (UniqueName: \"kubernetes.io/projected/9662917b-5681-4a8e-afe3-055f6ebcb23d-kube-api-access-q77t5\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.456932 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.456876 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9662917b-5681-4a8e-afe3-055f6ebcb23d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.456932 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.456905 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9662917b-5681-4a8e-afe3-055f6ebcb23d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.456932 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.456929 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9662917b-5681-4a8e-afe3-055f6ebcb23d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.457088 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.456974 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9662917b-5681-4a8e-afe3-055f6ebcb23d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.557472 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.557429 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9662917b-5681-4a8e-afe3-055f6ebcb23d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.557472 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.557474 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9662917b-5681-4a8e-afe3-055f6ebcb23d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.557708 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.557527 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9662917b-5681-4a8e-afe3-055f6ebcb23d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.557708 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.557559 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q77t5\" (UniqueName: \"kubernetes.io/projected/9662917b-5681-4a8e-afe3-055f6ebcb23d-kube-api-access-q77t5\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.557708 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.557595 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9662917b-5681-4a8e-afe3-055f6ebcb23d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.557708 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.557622 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9662917b-5681-4a8e-afe3-055f6ebcb23d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.557708 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.557653 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9662917b-5681-4a8e-afe3-055f6ebcb23d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.558510 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.558449 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9662917b-5681-4a8e-afe3-055f6ebcb23d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.560107 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.560073 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9662917b-5681-4a8e-afe3-055f6ebcb23d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.560228 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.560088 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9662917b-5681-4a8e-afe3-055f6ebcb23d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.560228 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.560199 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9662917b-5681-4a8e-afe3-055f6ebcb23d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.560366 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.560263 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9662917b-5681-4a8e-afe3-055f6ebcb23d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.574618 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.574586 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q77t5\" (UniqueName: \"kubernetes.io/projected/9662917b-5681-4a8e-afe3-055f6ebcb23d-kube-api-access-q77t5\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.574900 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.574882 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9662917b-5681-4a8e-afe3-055f6ebcb23d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ln4jb\" (UID: \"9662917b-5681-4a8e-afe3-055f6ebcb23d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.704026 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.703924 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:55.847375 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.847342 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb"] Jul 14 01:06:55.850655 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:06:55.850629 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9662917b_5681_4a8e_afe3_055f6ebcb23d.slice/crio-2849cd5de3c98634b676d8ea8b53bf4e5ee41bee38c2bf4812a912801f4af28b WatchSource:0}: Error finding container 2849cd5de3c98634b676d8ea8b53bf4e5ee41bee38c2bf4812a912801f4af28b: Status 404 returned error can't find the container with id 2849cd5de3c98634b676d8ea8b53bf4e5ee41bee38c2bf4812a912801f4af28b Jul 14 01:06:55.860045 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.860017 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdf94a72-0712-48e2-8131-a13bfcd34872-tls-certs\") pod \"kube-auth-proxy-54d956c5b-x5bv2\" (UID: \"fdf94a72-0712-48e2-8131-a13bfcd34872\") " pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" Jul 14 01:06:55.862530 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.862511 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdf94a72-0712-48e2-8131-a13bfcd34872-tls-certs\") pod \"kube-auth-proxy-54d956c5b-x5bv2\" (UID: \"fdf94a72-0712-48e2-8131-a13bfcd34872\") " pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" Jul 14 01:06:55.964526 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:55.964447 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" Jul 14 01:06:56.110578 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:56.110554 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2"] Jul 14 01:06:56.112462 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:06:56.112441 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdf94a72_0712_48e2_8131_a13bfcd34872.slice/crio-7118a7baa5a4703b2824f17c8be48cb1ddb06d6d16f32cf421bda53fedb0736e WatchSource:0}: Error finding container 7118a7baa5a4703b2824f17c8be48cb1ddb06d6d16f32cf421bda53fedb0736e: Status 404 returned error can't find the container with id 7118a7baa5a4703b2824f17c8be48cb1ddb06d6d16f32cf421bda53fedb0736e Jul 14 01:06:56.713440 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:56.713397 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" event={"ID":"fdf94a72-0712-48e2-8131-a13bfcd34872","Type":"ContainerStarted","Data":"7118a7baa5a4703b2824f17c8be48cb1ddb06d6d16f32cf421bda53fedb0736e"} Jul 14 01:06:56.715004 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:56.714971 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" event={"ID":"9662917b-5681-4a8e-afe3-055f6ebcb23d","Type":"ContainerStarted","Data":"2849cd5de3c98634b676d8ea8b53bf4e5ee41bee38c2bf4812a912801f4af28b"} Jul 14 01:06:59.271237 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:59.271196 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:06:59.271654 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:59.271265 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:06:59.731233 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:59.731201 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" event={"ID":"fdf94a72-0712-48e2-8131-a13bfcd34872","Type":"ContainerStarted","Data":"5102ace1b929c9c7784eb07b052bdd33fcca563d7bbb5202565e3223032b7867"} Jul 14 01:06:59.732847 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:59.732819 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" event={"ID":"9662917b-5681-4a8e-afe3-055f6ebcb23d","Type":"ContainerStarted","Data":"1a0b42e70e2b40577f7e2cbd7e096655a25521c5062c85c0f19e045760420bdb"} Jul 14 01:06:59.732965 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:59.732868 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:06:59.749193 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:59.749143 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-54d956c5b-x5bv2" podStartSLOduration=2.589084802 podStartE2EDuration="5.749125847s" totalImagesPullingTime="3.160041045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:06:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:06:59.747765396 +0000 UTC m=+448.650620895" watchObservedRunningTime="2026-07-14 01:06:59.749125847 +0000 UTC m=+448.651981347" Jul 14 01:06:59.766619 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:06:59.766557 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" podStartSLOduration=1.3482864669999999 podStartE2EDuration="4.766536091s" totalImagesPullingTime="3.418249624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:06:59.764865293 +0000 UTC m=+448.667720792" watchObservedRunningTime="2026-07-14 01:06:59.766536091 +0000 UTC m=+448.669391592" Jul 14 01:07:00.739548 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:00.739512 2582 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-ln4jb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 01:07:00.739920 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:00.739573 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" podUID="9662917b-5681-4a8e-afe3-055f6ebcb23d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 01:07:01.178399 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.178366 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f"] Jul 14 01:07:01.181742 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.181725 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.184501 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.184480 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ffqqb\"" Jul 14 01:07:01.196506 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.196484 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f"] Jul 14 01:07:01.311410 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.311370 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e18dac72-7f63-4046-9994-747715f7d4b2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.311410 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.311413 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e18dac72-7f63-4046-9994-747715f7d4b2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.311637 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.311483 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e18dac72-7f63-4046-9994-747715f7d4b2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.311637 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.311532 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e18dac72-7f63-4046-9994-747715f7d4b2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.311637 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.311552 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e18dac72-7f63-4046-9994-747715f7d4b2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.311637 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.311571 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e18dac72-7f63-4046-9994-747715f7d4b2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.311637 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.311628 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e18dac72-7f63-4046-9994-747715f7d4b2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.311938 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.311645 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e18dac72-7f63-4046-9994-747715f7d4b2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.311938 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.311694 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrn66\" (UniqueName: \"kubernetes.io/projected/e18dac72-7f63-4046-9994-747715f7d4b2-kube-api-access-jrn66\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.412211 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.412176 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e18dac72-7f63-4046-9994-747715f7d4b2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.412423 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.412228 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e18dac72-7f63-4046-9994-747715f7d4b2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.412423 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.412349 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e18dac72-7f63-4046-9994-747715f7d4b2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.412423 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.412370 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e18dac72-7f63-4046-9994-747715f7d4b2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.412423 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.412389 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e18dac72-7f63-4046-9994-747715f7d4b2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.412423 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.412424 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e18dac72-7f63-4046-9994-747715f7d4b2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.412695 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.412440 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e18dac72-7f63-4046-9994-747715f7d4b2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.412695 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.412463 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrn66\" (UniqueName: \"kubernetes.io/projected/e18dac72-7f63-4046-9994-747715f7d4b2-kube-api-access-jrn66\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.412695 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.412523 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e18dac72-7f63-4046-9994-747715f7d4b2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.419186 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.413086 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e18dac72-7f63-4046-9994-747715f7d4b2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.419186 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.413259 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e18dac72-7f63-4046-9994-747715f7d4b2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.419186 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.412715 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e18dac72-7f63-4046-9994-747715f7d4b2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.419186 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.413294 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e18dac72-7f63-4046-9994-747715f7d4b2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.419186 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.413556 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e18dac72-7f63-4046-9994-747715f7d4b2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.419186 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.415818 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e18dac72-7f63-4046-9994-747715f7d4b2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.421898 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.421876 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e18dac72-7f63-4046-9994-747715f7d4b2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.425434 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.425411 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e18dac72-7f63-4046-9994-747715f7d4b2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.426168 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.426143 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrn66\" (UniqueName: \"kubernetes.io/projected/e18dac72-7f63-4046-9994-747715f7d4b2-kube-api-access-jrn66\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4k6q4f\" (UID: \"e18dac72-7f63-4046-9994-747715f7d4b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.492198 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.492094 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:01.628586 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.628560 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f"] Jul 14 01:07:01.630454 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:07:01.630426 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode18dac72_7f63_4046_9994_747715f7d4b2.slice/crio-c88818ef9ec28f04afbccec0af52ab486c3a1251b495c8732219d8c7ccf371e9 WatchSource:0}: Error finding container c88818ef9ec28f04afbccec0af52ab486c3a1251b495c8732219d8c7ccf371e9: Status 404 returned error can't find the container with id c88818ef9ec28f04afbccec0af52ab486c3a1251b495c8732219d8c7ccf371e9 Jul 14 01:07:01.743203 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:01.743114 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" event={"ID":"e18dac72-7f63-4046-9994-747715f7d4b2","Type":"ContainerStarted","Data":"c88818ef9ec28f04afbccec0af52ab486c3a1251b495c8732219d8c7ccf371e9"} Jul 14 01:07:03.738779 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:03.738747 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln4jb" Jul 14 01:07:04.112591 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:04.112560 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:07:04.112682 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:04.112636 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:07:04.112682 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:04.112666 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:07:04.704670 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:04.704636 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-rp96x" Jul 14 01:07:04.706811 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:04.706784 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-422x7" Jul 14 01:07:04.761929 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:04.761887 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" event={"ID":"e18dac72-7f63-4046-9994-747715f7d4b2","Type":"ContainerStarted","Data":"59166be977a3669986d506d8c5a9d718662b87c10740bbd919d3bc97a1f24754"} Jul 14 01:07:04.782252 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:04.782201 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" podStartSLOduration=1.302440133 podStartE2EDuration="3.782184623s" totalImagesPullingTime="2.47974449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:07:04.780801071 +0000 UTC m=+453.683656571" watchObservedRunningTime="2026-07-14 01:07:04.782184623 +0000 UTC m=+453.685040122" Jul 14 01:07:05.493094 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:05.493059 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:05.498333 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:05.498292 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:05.766900 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:05.766818 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:07:05.767917 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:07:05.767899 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4k6q4f" Jul 14 01:08:24.344746 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:24.344662 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vzq6c"] Jul 14 01:08:24.347799 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:24.347782 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vzq6c" Jul 14 01:08:24.350479 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:24.350456 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 01:08:24.350575 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:24.350534 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 01:08:24.351498 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:24.351482 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-nlrnc\"" Jul 14 01:08:24.362014 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:24.361990 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vzq6c"] Jul 14 01:08:24.390938 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:24.390903 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcv56\" (UniqueName: \"kubernetes.io/projected/8cef83ac-b449-4ab8-a93e-a8edff34092f-kube-api-access-pcv56\") pod \"authorino-operator-75cc8d5cb7-vzq6c\" (UID: \"8cef83ac-b449-4ab8-a93e-a8edff34092f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vzq6c" Jul 14 01:08:24.491713 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:24.491678 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcv56\" (UniqueName: \"kubernetes.io/projected/8cef83ac-b449-4ab8-a93e-a8edff34092f-kube-api-access-pcv56\") pod \"authorino-operator-75cc8d5cb7-vzq6c\" (UID: \"8cef83ac-b449-4ab8-a93e-a8edff34092f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vzq6c" Jul 14 01:08:24.501541 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:24.501517 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcv56\" (UniqueName: \"kubernetes.io/projected/8cef83ac-b449-4ab8-a93e-a8edff34092f-kube-api-access-pcv56\") pod \"authorino-operator-75cc8d5cb7-vzq6c\" (UID: \"8cef83ac-b449-4ab8-a93e-a8edff34092f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vzq6c" Jul 14 01:08:24.658009 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:24.657923 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vzq6c" Jul 14 01:08:24.793784 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:24.793759 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vzq6c"] Jul 14 01:08:24.796189 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:08:24.796159 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cef83ac_b449_4ab8_a93e_a8edff34092f.slice/crio-29d52c0d72be28ccd298e7bb71e7033474196c1f3a8e6502c94a5b54593296a5 WatchSource:0}: Error finding container 29d52c0d72be28ccd298e7bb71e7033474196c1f3a8e6502c94a5b54593296a5: Status 404 returned error can't find the container with id 29d52c0d72be28ccd298e7bb71e7033474196c1f3a8e6502c94a5b54593296a5 Jul 14 01:08:25.124590 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:25.124549 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vzq6c" event={"ID":"8cef83ac-b449-4ab8-a93e-a8edff34092f","Type":"ContainerStarted","Data":"29d52c0d72be28ccd298e7bb71e7033474196c1f3a8e6502c94a5b54593296a5"} Jul 14 01:08:28.141465 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:28.141427 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vzq6c" event={"ID":"8cef83ac-b449-4ab8-a93e-a8edff34092f","Type":"ContainerStarted","Data":"30d6d98cab25f6fa2db42de539c762f625f04e1591ab6e65eaa8ec59732b5e6d"} Jul 14 01:08:28.141948 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:28.141564 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vzq6c" Jul 14 01:08:28.168781 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:28.168728 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vzq6c" podStartSLOduration=1.27917897 podStartE2EDuration="4.168713322s" totalImagesPullingTime="2.889534352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:08:28.167647508 +0000 UTC m=+537.070503008" watchObservedRunningTime="2026-07-14 01:08:28.168713322 +0000 UTC m=+537.071568820" Jul 14 01:08:39.148450 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:39.148414 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vzq6c" Jul 14 01:08:42.015592 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:42.015555 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd"] Jul 14 01:08:42.018926 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:42.018910 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" Jul 14 01:08:42.024871 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:42.024845 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-x947v\"" Jul 14 01:08:42.042565 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:42.042536 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd"] Jul 14 01:08:42.051770 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:42.051748 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-926xg\" (UniqueName: \"kubernetes.io/projected/24c090d3-216d-4abb-af9d-ad10a083df81-kube-api-access-926xg\") pod \"kuadrant-operator-controller-manager-86878c769f-cv2sd\" (UID: \"24c090d3-216d-4abb-af9d-ad10a083df81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" Jul 14 01:08:42.051903 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:42.051777 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24c090d3-216d-4abb-af9d-ad10a083df81-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cv2sd\" (UID: \"24c090d3-216d-4abb-af9d-ad10a083df81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" Jul 14 01:08:42.152682 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:42.152635 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-926xg\" (UniqueName: \"kubernetes.io/projected/24c090d3-216d-4abb-af9d-ad10a083df81-kube-api-access-926xg\") pod \"kuadrant-operator-controller-manager-86878c769f-cv2sd\" (UID: \"24c090d3-216d-4abb-af9d-ad10a083df81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" Jul 14 01:08:42.152682 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:42.152684 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24c090d3-216d-4abb-af9d-ad10a083df81-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cv2sd\" (UID: \"24c090d3-216d-4abb-af9d-ad10a083df81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" Jul 14 01:08:42.153132 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:42.153104 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24c090d3-216d-4abb-af9d-ad10a083df81-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cv2sd\" (UID: \"24c090d3-216d-4abb-af9d-ad10a083df81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" Jul 14 01:08:42.161112 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:42.161083 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-926xg\" (UniqueName: \"kubernetes.io/projected/24c090d3-216d-4abb-af9d-ad10a083df81-kube-api-access-926xg\") pod \"kuadrant-operator-controller-manager-86878c769f-cv2sd\" (UID: \"24c090d3-216d-4abb-af9d-ad10a083df81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" Jul 14 01:08:42.328710 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:42.328623 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" Jul 14 01:08:42.464951 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:42.464925 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd"] Jul 14 01:08:42.467045 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:08:42.467019 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c090d3_216d_4abb_af9d_ad10a083df81.slice/crio-c3a43be4939c366093d8a73866e38979d5c0d0854e71b80cf7d152606cebc95d WatchSource:0}: Error finding container c3a43be4939c366093d8a73866e38979d5c0d0854e71b80cf7d152606cebc95d: Status 404 returned error can't find the container with id c3a43be4939c366093d8a73866e38979d5c0d0854e71b80cf7d152606cebc95d Jul 14 01:08:43.213052 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:43.213013 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" event={"ID":"24c090d3-216d-4abb-af9d-ad10a083df81","Type":"ContainerStarted","Data":"c3a43be4939c366093d8a73866e38979d5c0d0854e71b80cf7d152606cebc95d"} Jul 14 01:08:48.240235 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:48.240148 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" event={"ID":"24c090d3-216d-4abb-af9d-ad10a083df81","Type":"ContainerStarted","Data":"bf24b495ae2fed441403f11a18755fbfbaced62d92e86eefea9df6184301bf12"} Jul 14 01:08:48.240235 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:48.240228 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" Jul 14 01:08:48.260070 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:48.260013 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" podStartSLOduration=1.739859573 podStartE2EDuration="7.259994611s" totalImagesPullingTime="5.520135038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:08:48.257706826 +0000 UTC m=+557.160562337" watchObservedRunningTime="2026-07-14 01:08:48.259994611 +0000 UTC m=+557.162850110" Jul 14 01:08:59.246849 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:08:59.246817 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" Jul 14 01:09:31.572368 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:31.572335 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:09:31.572906 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:31.572540 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:09:57.377408 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:57.377305 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c774b5cf7-9zjpc"] Jul 14 01:09:57.380837 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:57.380813 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" Jul 14 01:09:57.384257 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:57.384237 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 14 01:09:57.384390 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:57.384286 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dncbs\"" Jul 14 01:09:57.387971 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:57.387669 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c774b5cf7-9zjpc"] Jul 14 01:09:57.514497 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:57.514456 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4rzx\" (UniqueName: \"kubernetes.io/projected/ff7e4216-7208-46cc-8f11-e6e074cd19b3-kube-api-access-v4rzx\") pod \"authorino-7c774b5cf7-9zjpc\" (UID: \"ff7e4216-7208-46cc-8f11-e6e074cd19b3\") " pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" Jul 14 01:09:57.514678 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:57.514578 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ff7e4216-7208-46cc-8f11-e6e074cd19b3-tls-cert\") pod \"authorino-7c774b5cf7-9zjpc\" (UID: \"ff7e4216-7208-46cc-8f11-e6e074cd19b3\") " pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" Jul 14 01:09:57.615231 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:57.615191 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4rzx\" (UniqueName: \"kubernetes.io/projected/ff7e4216-7208-46cc-8f11-e6e074cd19b3-kube-api-access-v4rzx\") pod \"authorino-7c774b5cf7-9zjpc\" (UID: \"ff7e4216-7208-46cc-8f11-e6e074cd19b3\") " pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" Jul 14 01:09:57.615426 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:57.615252 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ff7e4216-7208-46cc-8f11-e6e074cd19b3-tls-cert\") pod \"authorino-7c774b5cf7-9zjpc\" (UID: \"ff7e4216-7208-46cc-8f11-e6e074cd19b3\") " pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" Jul 14 01:09:57.617922 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:57.617901 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ff7e4216-7208-46cc-8f11-e6e074cd19b3-tls-cert\") pod \"authorino-7c774b5cf7-9zjpc\" (UID: \"ff7e4216-7208-46cc-8f11-e6e074cd19b3\") " pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" Jul 14 01:09:57.622482 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:57.622458 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4rzx\" (UniqueName: \"kubernetes.io/projected/ff7e4216-7208-46cc-8f11-e6e074cd19b3-kube-api-access-v4rzx\") pod \"authorino-7c774b5cf7-9zjpc\" (UID: \"ff7e4216-7208-46cc-8f11-e6e074cd19b3\") " pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" Jul 14 01:09:57.691036 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:57.690942 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" Jul 14 01:09:57.823364 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:57.823334 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c774b5cf7-9zjpc"] Jul 14 01:09:57.825283 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:09:57.825260 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff7e4216_7208_46cc_8f11_e6e074cd19b3.slice/crio-fde275d6e80731900b1f0e83b21c8fe6edf4fb9eb5b1348fa403a9a8fa23e254 WatchSource:0}: Error finding container fde275d6e80731900b1f0e83b21c8fe6edf4fb9eb5b1348fa403a9a8fa23e254: Status 404 returned error can't find the container with id fde275d6e80731900b1f0e83b21c8fe6edf4fb9eb5b1348fa403a9a8fa23e254 Jul 14 01:09:58.575103 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:09:58.575070 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" event={"ID":"ff7e4216-7208-46cc-8f11-e6e074cd19b3","Type":"ContainerStarted","Data":"fde275d6e80731900b1f0e83b21c8fe6edf4fb9eb5b1348fa403a9a8fa23e254"} Jul 14 01:10:02.600017 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:10:02.599978 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" event={"ID":"ff7e4216-7208-46cc-8f11-e6e074cd19b3","Type":"ContainerStarted","Data":"84aa78e9fc3dc9599084756f978fd4cf5d33e06bf074821eefd2869010b0fdda"} Jul 14 01:10:02.615715 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:10:02.615662 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" podStartSLOduration=1.521373812 podStartE2EDuration="5.615645546s" totalImagesPullingTime="4.094271734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:10:02.614386325 +0000 UTC m=+631.517241825" watchObservedRunningTime="2026-07-14 01:10:02.615645546 +0000 UTC m=+631.518501044" Jul 14 01:11:33.828999 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:33.828909 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l"] Jul 14 01:11:33.832623 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:33.832599 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:33.836436 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:33.836415 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-48wgh\"" Jul 14 01:11:33.836436 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:33.836424 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 14 01:11:33.836631 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:33.836422 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 01:11:33.836631 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:33.836434 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 01:11:33.843508 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:33.843477 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l"] Jul 14 01:11:33.974528 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:33.974474 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:33.974701 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:33.974536 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:33.974701 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:33.974598 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:33.974701 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:33.974628 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcjxm\" (UniqueName: \"kubernetes.io/projected/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-kube-api-access-zcjxm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:33.974701 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:33.974677 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:33.974839 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:33.974720 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:34.075495 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:34.075452 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:34.075691 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:34.075511 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:34.075691 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:34.075541 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:34.075691 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:34.075564 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:34.075691 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:34.075602 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:34.075691 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:34.075619 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcjxm\" (UniqueName: \"kubernetes.io/projected/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-kube-api-access-zcjxm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:34.076070 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:34.076050 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:34.076145 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:34.076109 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:34.076201 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:34.076157 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:34.078051 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:34.078019 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:34.078164 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:34.078122 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:34.084649 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:34.084620 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcjxm\" (UniqueName: \"kubernetes.io/projected/bb66014b-51f0-48b1-972b-5aa42c9d6fb4-kube-api-access-zcjxm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9c42l\" (UID: \"bb66014b-51f0-48b1-972b-5aa42c9d6fb4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:34.144349 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:34.144273 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:34.284368 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:34.284341 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l"] Jul 14 01:11:34.286244 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:11:34.286215 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb66014b_51f0_48b1_972b_5aa42c9d6fb4.slice/crio-b66820d933b5d95337793ef8d0a47f3a4528952f2b8e5d1c79bb49c3777406a6 WatchSource:0}: Error finding container b66820d933b5d95337793ef8d0a47f3a4528952f2b8e5d1c79bb49c3777406a6: Status 404 returned error can't find the container with id b66820d933b5d95337793ef8d0a47f3a4528952f2b8e5d1c79bb49c3777406a6 Jul 14 01:11:34.287651 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:34.287631 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 01:11:35.033334 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:35.033290 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" event={"ID":"bb66014b-51f0-48b1-972b-5aa42c9d6fb4","Type":"ContainerStarted","Data":"b66820d933b5d95337793ef8d0a47f3a4528952f2b8e5d1c79bb49c3777406a6"} Jul 14 01:11:36.040201 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:36.040158 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" event={"ID":"bb66014b-51f0-48b1-972b-5aa42c9d6fb4","Type":"ContainerStarted","Data":"ba2ed3a0c4addae35cd5670558983da475eda2e973651f3ef949f00bd44ae5a2"} Jul 14 01:11:36.058298 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:36.058225 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" podStartSLOduration=2.030652903 podStartE2EDuration="3.058203404s" totalImagesPullingTime="1.027550501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:11:36.055811672 +0000 UTC m=+724.958667175" watchObservedRunningTime="2026-07-14 01:11:36.058203404 +0000 UTC m=+724.961058905" Jul 14 01:11:44.144778 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:44.144742 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:44.145178 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:44.145047 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:44.157611 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:44.157587 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:45.093631 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:45.093602 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9c42l" Jul 14 01:11:57.437314 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.437276 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd"] Jul 14 01:11:57.440564 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.440543 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.443236 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.443214 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 14 01:11:57.449643 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.449613 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd"] Jul 14 01:11:57.582714 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.582682 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e0d08e0-7561-4713-beff-1962e9415d0c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.582882 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.582727 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pts94\" (UniqueName: \"kubernetes.io/projected/1e0d08e0-7561-4713-beff-1962e9415d0c-kube-api-access-pts94\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.582882 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.582761 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e0d08e0-7561-4713-beff-1962e9415d0c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.582882 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.582779 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e0d08e0-7561-4713-beff-1962e9415d0c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.582882 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.582802 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e0d08e0-7561-4713-beff-1962e9415d0c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.583019 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.582907 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e0d08e0-7561-4713-beff-1962e9415d0c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.683977 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.683940 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e0d08e0-7561-4713-beff-1962e9415d0c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.684167 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.684012 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e0d08e0-7561-4713-beff-1962e9415d0c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.684167 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.684029 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pts94\" (UniqueName: \"kubernetes.io/projected/1e0d08e0-7561-4713-beff-1962e9415d0c-kube-api-access-pts94\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.684167 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.684054 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e0d08e0-7561-4713-beff-1962e9415d0c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.684167 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.684072 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e0d08e0-7561-4713-beff-1962e9415d0c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.684167 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.684088 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e0d08e0-7561-4713-beff-1962e9415d0c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.684468 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.684445 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e0d08e0-7561-4713-beff-1962e9415d0c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.684511 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.684457 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e0d08e0-7561-4713-beff-1962e9415d0c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.684583 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.684559 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e0d08e0-7561-4713-beff-1962e9415d0c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.686660 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.686633 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e0d08e0-7561-4713-beff-1962e9415d0c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.686898 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.686881 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e0d08e0-7561-4713-beff-1962e9415d0c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.692671 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.692612 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pts94\" (UniqueName: \"kubernetes.io/projected/1e0d08e0-7561-4713-beff-1962e9415d0c-kube-api-access-pts94\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trvdd\" (UID: \"1e0d08e0-7561-4713-beff-1962e9415d0c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.751954 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.751906 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:11:57.888986 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:57.888959 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd"] Jul 14 01:11:57.890942 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:11:57.890914 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e0d08e0_7561_4713_beff_1962e9415d0c.slice/crio-5bbbd0c7507f474bff17801bd4916a40eab97c430d9e7826d7c0df8e42594ff7 WatchSource:0}: Error finding container 5bbbd0c7507f474bff17801bd4916a40eab97c430d9e7826d7c0df8e42594ff7: Status 404 returned error can't find the container with id 5bbbd0c7507f474bff17801bd4916a40eab97c430d9e7826d7c0df8e42594ff7 Jul 14 01:11:58.150371 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:58.150316 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" event={"ID":"1e0d08e0-7561-4713-beff-1962e9415d0c","Type":"ContainerStarted","Data":"5bbbd0c7507f474bff17801bd4916a40eab97c430d9e7826d7c0df8e42594ff7"} Jul 14 01:11:59.156195 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:59.156156 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" event={"ID":"1e0d08e0-7561-4713-beff-1962e9415d0c","Type":"ContainerStarted","Data":"b7545dfc41ca15ce8454e593c35195a4f3db28688b5a76f021b210701758d9a9"} Jul 14 01:11:59.174734 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:11:59.174677 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" podStartSLOduration=1.9508751370000001 podStartE2EDuration="2.17465959s" totalImagesPullingTime="223.784453ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:11:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:11:59.172849742 +0000 UTC m=+748.075705252" watchObservedRunningTime="2026-07-14 01:11:59.17465959 +0000 UTC m=+748.077515090" Jul 14 01:12:07.753009 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:12:07.752972 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:12:07.753524 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:12:07.753040 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:12:07.765359 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:12:07.765334 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:12:08.211703 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:12:08.211674 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trvdd" Jul 14 01:13:06.519547 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:06.519506 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c774b5cf7-9zjpc"] Jul 14 01:13:06.520144 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:06.519745 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" podUID="ff7e4216-7208-46cc-8f11-e6e074cd19b3" containerName="authorino" containerID="cri-o://84aa78e9fc3dc9599084756f978fd4cf5d33e06bf074821eefd2869010b0fdda" gracePeriod=30 Jul 14 01:13:06.778119 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:06.778049 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" Jul 14 01:13:06.817888 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:06.817855 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ff7e4216-7208-46cc-8f11-e6e074cd19b3-tls-cert\") pod \"ff7e4216-7208-46cc-8f11-e6e074cd19b3\" (UID: \"ff7e4216-7208-46cc-8f11-e6e074cd19b3\") " Jul 14 01:13:06.818051 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:06.817915 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4rzx\" (UniqueName: \"kubernetes.io/projected/ff7e4216-7208-46cc-8f11-e6e074cd19b3-kube-api-access-v4rzx\") pod \"ff7e4216-7208-46cc-8f11-e6e074cd19b3\" (UID: \"ff7e4216-7208-46cc-8f11-e6e074cd19b3\") " Jul 14 01:13:06.820505 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:06.820468 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff7e4216-7208-46cc-8f11-e6e074cd19b3-kube-api-access-v4rzx" (OuterVolumeSpecName: "kube-api-access-v4rzx") pod "ff7e4216-7208-46cc-8f11-e6e074cd19b3" (UID: "ff7e4216-7208-46cc-8f11-e6e074cd19b3"). InnerVolumeSpecName "kube-api-access-v4rzx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:13:06.830313 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:06.830272 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff7e4216-7208-46cc-8f11-e6e074cd19b3-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ff7e4216-7208-46cc-8f11-e6e074cd19b3" (UID: "ff7e4216-7208-46cc-8f11-e6e074cd19b3"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 01:13:06.918792 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:06.918752 2582 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ff7e4216-7208-46cc-8f11-e6e074cd19b3-tls-cert\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:13:06.918974 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:06.918810 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4rzx\" (UniqueName: \"kubernetes.io/projected/ff7e4216-7208-46cc-8f11-e6e074cd19b3-kube-api-access-v4rzx\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:13:07.496419 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:07.496381 2582 generic.go:356] "Generic (PLEG): container finished" podID="ff7e4216-7208-46cc-8f11-e6e074cd19b3" containerID="84aa78e9fc3dc9599084756f978fd4cf5d33e06bf074821eefd2869010b0fdda" exitCode=0 Jul 14 01:13:07.496618 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:07.496439 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" Jul 14 01:13:07.496618 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:07.496483 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" event={"ID":"ff7e4216-7208-46cc-8f11-e6e074cd19b3","Type":"ContainerDied","Data":"84aa78e9fc3dc9599084756f978fd4cf5d33e06bf074821eefd2869010b0fdda"} Jul 14 01:13:07.496618 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:07.496536 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c774b5cf7-9zjpc" event={"ID":"ff7e4216-7208-46cc-8f11-e6e074cd19b3","Type":"ContainerDied","Data":"fde275d6e80731900b1f0e83b21c8fe6edf4fb9eb5b1348fa403a9a8fa23e254"} Jul 14 01:13:07.496618 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:07.496557 2582 scope.go:117] "RemoveContainer" containerID="84aa78e9fc3dc9599084756f978fd4cf5d33e06bf074821eefd2869010b0fdda" Jul 14 01:13:07.505675 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:07.505656 2582 scope.go:117] "RemoveContainer" containerID="84aa78e9fc3dc9599084756f978fd4cf5d33e06bf074821eefd2869010b0fdda" Jul 14 01:13:07.505967 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:13:07.505950 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84aa78e9fc3dc9599084756f978fd4cf5d33e06bf074821eefd2869010b0fdda\": container with ID starting with 84aa78e9fc3dc9599084756f978fd4cf5d33e06bf074821eefd2869010b0fdda not found: ID does not exist" containerID="84aa78e9fc3dc9599084756f978fd4cf5d33e06bf074821eefd2869010b0fdda" Jul 14 01:13:07.506030 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:07.505980 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84aa78e9fc3dc9599084756f978fd4cf5d33e06bf074821eefd2869010b0fdda"} err="failed to get container status \"84aa78e9fc3dc9599084756f978fd4cf5d33e06bf074821eefd2869010b0fdda\": rpc error: code = NotFound desc = could not find container \"84aa78e9fc3dc9599084756f978fd4cf5d33e06bf074821eefd2869010b0fdda\": container with ID starting with 84aa78e9fc3dc9599084756f978fd4cf5d33e06bf074821eefd2869010b0fdda not found: ID does not exist" Jul 14 01:13:07.516964 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:07.516929 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c774b5cf7-9zjpc"] Jul 14 01:13:07.520553 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:07.520530 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7c774b5cf7-9zjpc"] Jul 14 01:13:07.693258 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:13:07.693213 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff7e4216-7208-46cc-8f11-e6e074cd19b3" path="/var/lib/kubelet/pods/ff7e4216-7208-46cc-8f11-e6e074cd19b3/volumes" Jul 14 01:14:31.601870 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:14:31.601842 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:14:31.602899 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:14:31.602878 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:15:00.141258 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:00.141213 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733195-w27m6"] Jul 14 01:15:00.141885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:00.141746 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff7e4216-7208-46cc-8f11-e6e074cd19b3" containerName="authorino" Jul 14 01:15:00.141885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:00.141769 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff7e4216-7208-46cc-8f11-e6e074cd19b3" containerName="authorino" Jul 14 01:15:00.141885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:00.141862 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff7e4216-7208-46cc-8f11-e6e074cd19b3" containerName="authorino" Jul 14 01:15:00.144953 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:00.144930 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733195-w27m6" Jul 14 01:15:00.147278 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:00.147259 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zn7l8\"" Jul 14 01:15:00.151201 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:00.151173 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733195-w27m6"] Jul 14 01:15:00.163217 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:00.163190 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733195-w27m6" Jul 14 01:15:00.302260 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:00.302235 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733195-w27m6"] Jul 14 01:15:00.304515 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:15:00.304485 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod886c16a7_5646_463b_91b3_d5a6144aeee5.slice/crio-06d3704effc55b264cc9975aaeafae665c26df4f6889df06cae454431b4fb746 WatchSource:0}: Error finding container 06d3704effc55b264cc9975aaeafae665c26df4f6889df06cae454431b4fb746: Status 404 returned error can't find the container with id 06d3704effc55b264cc9975aaeafae665c26df4f6889df06cae454431b4fb746 Jul 14 01:15:01.044028 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:01.043986 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733195-w27m6" event={"ID":"886c16a7-5646-463b-91b3-d5a6144aeee5","Type":"ContainerStarted","Data":"06d3704effc55b264cc9975aaeafae665c26df4f6889df06cae454431b4fb746"} Jul 14 01:15:03.057030 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:03.056940 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733195-w27m6" event={"ID":"886c16a7-5646-463b-91b3-d5a6144aeee5","Type":"ContainerStarted","Data":"f848b06809bae50230e353696cc1ea48bd1dca45ad6e82f3b0fc5921f960f4c2"} Jul 14 01:15:03.071919 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:03.071866 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733195-w27m6" podStartSLOduration=0.948387351 podStartE2EDuration="3.071849776s" totalImagesPullingTime="2.123462425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:15:03.070031757 +0000 UTC m=+931.972887256" watchObservedRunningTime="2026-07-14 01:15:03.071849776 +0000 UTC m=+931.974705276" Jul 14 01:15:04.063148 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:04.063113 2582 generic.go:356] "Generic (PLEG): container finished" podID="886c16a7-5646-463b-91b3-d5a6144aeee5" containerID="f848b06809bae50230e353696cc1ea48bd1dca45ad6e82f3b0fc5921f960f4c2" exitCode=0 Jul 14 01:15:04.063590 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:04.063204 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733195-w27m6" event={"ID":"886c16a7-5646-463b-91b3-d5a6144aeee5","Type":"ContainerDied","Data":"f848b06809bae50230e353696cc1ea48bd1dca45ad6e82f3b0fc5921f960f4c2"} Jul 14 01:15:05.207507 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:05.207480 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733195-w27m6" Jul 14 01:15:06.075605 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:06.075513 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733195-w27m6" Jul 14 01:15:06.075605 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:06.075553 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733195-w27m6" event={"ID":"886c16a7-5646-463b-91b3-d5a6144aeee5","Type":"ContainerDied","Data":"06d3704effc55b264cc9975aaeafae665c26df4f6889df06cae454431b4fb746"} Jul 14 01:15:06.075793 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:15:06.075608 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06d3704effc55b264cc9975aaeafae665c26df4f6889df06cae454431b4fb746" Jul 14 01:19:13.018691 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:13.018650 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd"] Jul 14 01:19:13.019232 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:13.018862 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" podUID="24c090d3-216d-4abb-af9d-ad10a083df81" containerName="manager" containerID="cri-o://bf24b495ae2fed441403f11a18755fbfbaced62d92e86eefea9df6184301bf12" gracePeriod=10 Jul 14 01:19:15.477196 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:15.477165 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" Jul 14 01:19:15.549065 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:15.548985 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24c090d3-216d-4abb-af9d-ad10a083df81-extensions-socket-volume\") pod \"24c090d3-216d-4abb-af9d-ad10a083df81\" (UID: \"24c090d3-216d-4abb-af9d-ad10a083df81\") " Jul 14 01:19:15.549065 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:15.549058 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-926xg\" (UniqueName: \"kubernetes.io/projected/24c090d3-216d-4abb-af9d-ad10a083df81-kube-api-access-926xg\") pod \"24c090d3-216d-4abb-af9d-ad10a083df81\" (UID: \"24c090d3-216d-4abb-af9d-ad10a083df81\") " Jul 14 01:19:15.549498 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:15.549456 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24c090d3-216d-4abb-af9d-ad10a083df81-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "24c090d3-216d-4abb-af9d-ad10a083df81" (UID: "24c090d3-216d-4abb-af9d-ad10a083df81"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:19:15.551270 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:15.551250 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24c090d3-216d-4abb-af9d-ad10a083df81-kube-api-access-926xg" (OuterVolumeSpecName: "kube-api-access-926xg") pod "24c090d3-216d-4abb-af9d-ad10a083df81" (UID: "24c090d3-216d-4abb-af9d-ad10a083df81"). InnerVolumeSpecName "kube-api-access-926xg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:19:15.649922 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:15.649886 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-926xg\" (UniqueName: \"kubernetes.io/projected/24c090d3-216d-4abb-af9d-ad10a083df81-kube-api-access-926xg\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:19:15.649922 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:15.649914 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24c090d3-216d-4abb-af9d-ad10a083df81-extensions-socket-volume\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:19:16.300080 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:16.300046 2582 generic.go:356] "Generic (PLEG): container finished" podID="24c090d3-216d-4abb-af9d-ad10a083df81" containerID="bf24b495ae2fed441403f11a18755fbfbaced62d92e86eefea9df6184301bf12" exitCode=0 Jul 14 01:19:16.300279 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:16.300092 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" Jul 14 01:19:16.300279 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:16.300139 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" event={"ID":"24c090d3-216d-4abb-af9d-ad10a083df81","Type":"ContainerDied","Data":"bf24b495ae2fed441403f11a18755fbfbaced62d92e86eefea9df6184301bf12"} Jul 14 01:19:16.300279 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:16.300201 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd" event={"ID":"24c090d3-216d-4abb-af9d-ad10a083df81","Type":"ContainerDied","Data":"c3a43be4939c366093d8a73866e38979d5c0d0854e71b80cf7d152606cebc95d"} Jul 14 01:19:16.300279 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:16.300230 2582 scope.go:117] "RemoveContainer" containerID="bf24b495ae2fed441403f11a18755fbfbaced62d92e86eefea9df6184301bf12" Jul 14 01:19:16.309830 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:16.309810 2582 scope.go:117] "RemoveContainer" containerID="bf24b495ae2fed441403f11a18755fbfbaced62d92e86eefea9df6184301bf12" Jul 14 01:19:16.310108 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:19:16.310086 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf24b495ae2fed441403f11a18755fbfbaced62d92e86eefea9df6184301bf12\": container with ID starting with bf24b495ae2fed441403f11a18755fbfbaced62d92e86eefea9df6184301bf12 not found: ID does not exist" containerID="bf24b495ae2fed441403f11a18755fbfbaced62d92e86eefea9df6184301bf12" Jul 14 01:19:16.310197 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:16.310114 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf24b495ae2fed441403f11a18755fbfbaced62d92e86eefea9df6184301bf12"} err="failed to get container status \"bf24b495ae2fed441403f11a18755fbfbaced62d92e86eefea9df6184301bf12\": rpc error: code = NotFound desc = could not find container \"bf24b495ae2fed441403f11a18755fbfbaced62d92e86eefea9df6184301bf12\": container with ID starting with bf24b495ae2fed441403f11a18755fbfbaced62d92e86eefea9df6184301bf12 not found: ID does not exist" Jul 14 01:19:16.317963 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:16.317939 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd"] Jul 14 01:19:16.321128 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:16.321103 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cv2sd"] Jul 14 01:19:17.693671 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:17.693640 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24c090d3-216d-4abb-af9d-ad10a083df81" path="/var/lib/kubelet/pods/24c090d3-216d-4abb-af9d-ad10a083df81/volumes" Jul 14 01:19:31.631610 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:31.631581 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:19:31.632888 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:19:31.632862 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:20:14.177432 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.177388 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6"] Jul 14 01:20:14.178405 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.177902 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="886c16a7-5646-463b-91b3-d5a6144aeee5" containerName="cleanup" Jul 14 01:20:14.178405 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.177921 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="886c16a7-5646-463b-91b3-d5a6144aeee5" containerName="cleanup" Jul 14 01:20:14.178405 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.177936 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24c090d3-216d-4abb-af9d-ad10a083df81" containerName="manager" Jul 14 01:20:14.178405 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.177944 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="24c090d3-216d-4abb-af9d-ad10a083df81" containerName="manager" Jul 14 01:20:14.178405 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.178041 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="886c16a7-5646-463b-91b3-d5a6144aeee5" containerName="cleanup" Jul 14 01:20:14.178405 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.178054 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="24c090d3-216d-4abb-af9d-ad10a083df81" containerName="manager" Jul 14 01:20:14.181194 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.181172 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6" Jul 14 01:20:14.184281 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.184262 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-x947v\"" Jul 14 01:20:14.193071 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.193045 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6"] Jul 14 01:20:14.258794 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.258757 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mcmb\" (UniqueName: \"kubernetes.io/projected/c22fd79d-ff1d-4db3-b674-3d51a4fa47ea-kube-api-access-4mcmb\") pod \"kuadrant-operator-controller-manager-86878c769f-xbtm6\" (UID: \"c22fd79d-ff1d-4db3-b674-3d51a4fa47ea\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6" Jul 14 01:20:14.258965 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.258805 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c22fd79d-ff1d-4db3-b674-3d51a4fa47ea-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xbtm6\" (UID: \"c22fd79d-ff1d-4db3-b674-3d51a4fa47ea\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6" Jul 14 01:20:14.359679 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.359622 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mcmb\" (UniqueName: \"kubernetes.io/projected/c22fd79d-ff1d-4db3-b674-3d51a4fa47ea-kube-api-access-4mcmb\") pod \"kuadrant-operator-controller-manager-86878c769f-xbtm6\" (UID: \"c22fd79d-ff1d-4db3-b674-3d51a4fa47ea\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6" Jul 14 01:20:14.359679 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.359682 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c22fd79d-ff1d-4db3-b674-3d51a4fa47ea-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xbtm6\" (UID: \"c22fd79d-ff1d-4db3-b674-3d51a4fa47ea\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6" Jul 14 01:20:14.360113 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.360093 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c22fd79d-ff1d-4db3-b674-3d51a4fa47ea-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xbtm6\" (UID: \"c22fd79d-ff1d-4db3-b674-3d51a4fa47ea\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6" Jul 14 01:20:14.370058 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.370028 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mcmb\" (UniqueName: \"kubernetes.io/projected/c22fd79d-ff1d-4db3-b674-3d51a4fa47ea-kube-api-access-4mcmb\") pod \"kuadrant-operator-controller-manager-86878c769f-xbtm6\" (UID: \"c22fd79d-ff1d-4db3-b674-3d51a4fa47ea\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6" Jul 14 01:20:14.491300 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.491207 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6" Jul 14 01:20:14.632972 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:14.632950 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6"] Jul 14 01:20:14.635994 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:20:14.635969 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc22fd79d_ff1d_4db3_b674_3d51a4fa47ea.slice/crio-7ebf8765b5ceffa279148d4918762c5211853ee3abdb64f2f501e2a595bb2aef WatchSource:0}: Error finding container 7ebf8765b5ceffa279148d4918762c5211853ee3abdb64f2f501e2a595bb2aef: Status 404 returned error can't find the container with id 7ebf8765b5ceffa279148d4918762c5211853ee3abdb64f2f501e2a595bb2aef Jul 14 01:20:15.586469 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:15.586432 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6" event={"ID":"c22fd79d-ff1d-4db3-b674-3d51a4fa47ea","Type":"ContainerStarted","Data":"1c44d16b618a3a344f121c991a6b79971a579c71b700e24cf4b65979505bf35c"} Jul 14 01:20:15.586839 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:15.586493 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6" event={"ID":"c22fd79d-ff1d-4db3-b674-3d51a4fa47ea","Type":"ContainerStarted","Data":"7ebf8765b5ceffa279148d4918762c5211853ee3abdb64f2f501e2a595bb2aef"} Jul 14 01:20:15.586839 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:15.586592 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6" Jul 14 01:20:15.606126 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:15.606081 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6" podStartSLOduration=1.606066062 podStartE2EDuration="1.606066062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:20:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:20:15.603642926 +0000 UTC m=+1244.506498427" watchObservedRunningTime="2026-07-14 01:20:15.606066062 +0000 UTC m=+1244.508921560" Jul 14 01:20:26.594136 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:20:26.594105 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xbtm6" Jul 14 01:24:31.661844 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:24:31.661818 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:24:31.663640 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:24:31.663613 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:26:24.979376 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:24.979288 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt"] Jul 14 01:26:24.982937 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:24.982906 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:24.985298 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:24.985273 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c626eb13-openshift-default-dockercfg-pgfr2\"" Jul 14 01:26:25.000996 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.000971 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt"] Jul 14 01:26:25.096683 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.096648 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6psnj\" (UniqueName: \"kubernetes.io/projected/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-kube-api-access-6psnj\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.096858 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.096687 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-workload-socket\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.096858 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.096709 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-podinfo\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.096858 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.096773 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-token\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.096858 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.096815 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-workload-certs\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.096858 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.096836 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istiod-ca-cert\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.097098 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.096909 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-envoy\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.097098 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.096939 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-data\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.097098 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.097011 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-credential-socket\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.198225 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.198187 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istiod-ca-cert\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.198430 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.198257 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-envoy\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.198430 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.198278 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-data\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.198430 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.198305 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-credential-socket\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.198430 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.198370 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6psnj\" (UniqueName: \"kubernetes.io/projected/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-kube-api-access-6psnj\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.198430 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.198399 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-workload-socket\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.198430 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.198417 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-podinfo\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.198699 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.198444 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-token\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.198699 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.198477 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-workload-certs\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.198826 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.198797 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-credential-socket\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.198985 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.198950 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-workload-socket\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.199078 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.199047 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-data\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.199078 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.199061 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istiod-ca-cert\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.199363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.199152 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-workload-certs\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.200857 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.200837 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-envoy\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.201083 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.201064 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-podinfo\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.206514 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.206491 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-token\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.206599 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.206571 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6psnj\" (UniqueName: \"kubernetes.io/projected/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-kube-api-access-6psnj\") pod \"e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.293671 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.293581 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:25.431916 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.431888 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt"] Jul 14 01:26:25.433234 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:26:25.433200 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82c369da_39cc_4c1c_8cd0_a08f8f98c1c2.slice/crio-b87e1bf9427bee1996d22944f90de4c82bb36ed908d3266bd474414e02806388 WatchSource:0}: Error finding container b87e1bf9427bee1996d22944f90de4c82bb36ed908d3266bd474414e02806388: Status 404 returned error can't find the container with id b87e1bf9427bee1996d22944f90de4c82bb36ed908d3266bd474414e02806388 Jul 14 01:26:25.435267 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.435235 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:26:25.435387 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.435310 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:26:25.435456 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:25.435392 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:26:26.403885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:26.403843 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" event={"ID":"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2","Type":"ContainerStarted","Data":"6f6be05afdfe2c80d2bfb16c383b0800e662ae5c221a1ebfc6fda5b1c7aa263f"} Jul 14 01:26:26.404351 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:26.403914 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" event={"ID":"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2","Type":"ContainerStarted","Data":"b87e1bf9427bee1996d22944f90de4c82bb36ed908d3266bd474414e02806388"} Jul 14 01:26:26.426134 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:26.426080 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" podStartSLOduration=2.426063582 podStartE2EDuration="2.426063582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:26:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:26:26.423724943 +0000 UTC m=+1615.326580442" watchObservedRunningTime="2026-07-14 01:26:26.426063582 +0000 UTC m=+1615.328919080" Jul 14 01:26:27.294016 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:27.293975 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:27.298550 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:27.298525 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:27.409025 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:27.408989 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:27.410038 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:27.410013 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:31.281555 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:31.281519 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd"] Jul 14 01:26:31.284895 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:31.284875 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd" Jul 14 01:26:31.287257 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:31.287229 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 01:26:31.287395 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:31.287379 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zn7l8\"" Jul 14 01:26:31.292480 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:31.292453 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd"] Jul 14 01:26:31.457288 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:31.457249 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85fbd4c9-f768-4e18-abc9-186632b6688a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd\" (UID: \"85fbd4c9-f768-4e18-abc9-186632b6688a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd" Jul 14 01:26:31.558357 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:31.558232 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85fbd4c9-f768-4e18-abc9-186632b6688a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd\" (UID: \"85fbd4c9-f768-4e18-abc9-186632b6688a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd" Jul 14 01:26:31.560667 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:31.560642 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 01:26:31.569279 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:31.569251 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85fbd4c9-f768-4e18-abc9-186632b6688a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd\" (UID: \"85fbd4c9-f768-4e18-abc9-186632b6688a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd" Jul 14 01:26:31.600136 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:31.600104 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zn7l8\"" Jul 14 01:26:31.607986 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:31.607958 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd" Jul 14 01:26:31.750119 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:31.750087 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd"] Jul 14 01:26:31.751756 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:26:31.751730 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85fbd4c9_f768_4e18_abc9_186632b6688a.slice/crio-b4e8d86a1249b005d9ecd34d263fc3fd3a79d961fc14cd40af527e98ff88931f WatchSource:0}: Error finding container b4e8d86a1249b005d9ecd34d263fc3fd3a79d961fc14cd40af527e98ff88931f: Status 404 returned error can't find the container with id b4e8d86a1249b005d9ecd34d263fc3fd3a79d961fc14cd40af527e98ff88931f Jul 14 01:26:32.436460 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:32.436417 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd" event={"ID":"85fbd4c9-f768-4e18-abc9-186632b6688a","Type":"ContainerStarted","Data":"86a83dc094649bb270f887e49a397e1de6ec2e23e698d5f2d50361d69494244d"} Jul 14 01:26:32.436844 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:32.436483 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd" event={"ID":"85fbd4c9-f768-4e18-abc9-186632b6688a","Type":"ContainerStarted","Data":"b4e8d86a1249b005d9ecd34d263fc3fd3a79d961fc14cd40af527e98ff88931f"} Jul 14 01:26:32.451162 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:32.451114 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd" podStartSLOduration=1.451097876 podStartE2EDuration="1.451097876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:26:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:26:32.449754069 +0000 UTC m=+1621.352609572" watchObservedRunningTime="2026-07-14 01:26:32.451097876 +0000 UTC m=+1621.353953411" Jul 14 01:26:33.442660 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:33.442625 2582 generic.go:356] "Generic (PLEG): container finished" podID="85fbd4c9-f768-4e18-abc9-186632b6688a" containerID="86a83dc094649bb270f887e49a397e1de6ec2e23e698d5f2d50361d69494244d" exitCode=7 Jul 14 01:26:33.443045 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:33.442716 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd" event={"ID":"85fbd4c9-f768-4e18-abc9-186632b6688a","Type":"ContainerDied","Data":"86a83dc094649bb270f887e49a397e1de6ec2e23e698d5f2d50361d69494244d"} Jul 14 01:26:33.443045 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:33.442901 2582 scope.go:117] "RemoveContainer" containerID="86a83dc094649bb270f887e49a397e1de6ec2e23e698d5f2d50361d69494244d" Jul 14 01:26:34.450062 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:34.450024 2582 generic.go:356] "Generic (PLEG): container finished" podID="85fbd4c9-f768-4e18-abc9-186632b6688a" containerID="cbc657b2a5abd20bcce36efd51930061a0dea6ecdd169829e98f56b31cc6181d" exitCode=0 Jul 14 01:26:34.450499 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:34.450077 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd" event={"ID":"85fbd4c9-f768-4e18-abc9-186632b6688a","Type":"ContainerDied","Data":"cbc657b2a5abd20bcce36efd51930061a0dea6ecdd169829e98f56b31cc6181d"} Jul 14 01:26:34.450499 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:34.450155 2582 scope.go:117] "RemoveContainer" containerID="86a83dc094649bb270f887e49a397e1de6ec2e23e698d5f2d50361d69494244d" Jul 14 01:26:35.632155 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:35.632132 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd" Jul 14 01:26:35.808549 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:35.808439 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85fbd4c9-f768-4e18-abc9-186632b6688a-maas-api-service-ca\") pod \"85fbd4c9-f768-4e18-abc9-186632b6688a\" (UID: \"85fbd4c9-f768-4e18-abc9-186632b6688a\") " Jul 14 01:26:35.808788 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:35.808764 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85fbd4c9-f768-4e18-abc9-186632b6688a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "85fbd4c9-f768-4e18-abc9-186632b6688a" (UID: "85fbd4c9-f768-4e18-abc9-186632b6688a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:26:35.909658 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:35.909618 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85fbd4c9-f768-4e18-abc9-186632b6688a-maas-api-service-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:26:36.461615 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:36.461585 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd" Jul 14 01:26:36.461615 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:36.461597 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd" event={"ID":"85fbd4c9-f768-4e18-abc9-186632b6688a","Type":"ContainerDied","Data":"b4e8d86a1249b005d9ecd34d263fc3fd3a79d961fc14cd40af527e98ff88931f"} Jul 14 01:26:36.461823 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:36.461673 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4e8d86a1249b005d9ecd34d263fc3fd3a79d961fc14cd40af527e98ff88931f" Jul 14 01:26:42.410590 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:42.410546 2582 patch_prober.go:28] interesting pod/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 14 01:26:42.411084 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:42.410626 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" podUID="82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 14 01:26:42.494129 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:42.494092 2582 generic.go:356] "Generic (PLEG): container finished" podID="82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" containerID="6f6be05afdfe2c80d2bfb16c383b0800e662ae5c221a1ebfc6fda5b1c7aa263f" exitCode=0 Jul 14 01:26:42.494296 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:42.494172 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" event={"ID":"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2","Type":"ContainerDied","Data":"6f6be05afdfe2c80d2bfb16c383b0800e662ae5c221a1ebfc6fda5b1c7aa263f"} Jul 14 01:26:42.494728 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:42.494709 2582 scope.go:117] "RemoveContainer" containerID="6f6be05afdfe2c80d2bfb16c383b0800e662ae5c221a1ebfc6fda5b1c7aa263f" Jul 14 01:26:42.495831 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:42.495801 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:26:42.495902 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:42.495884 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:26:42.495952 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:42.495938 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:26:43.505408 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:43.505365 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" event={"ID":"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2","Type":"ContainerStarted","Data":"284030cc0174f7f75316bfc144eed891c4af048295a09b63f30b6c75d0eed632"} Jul 14 01:26:44.294298 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:44.294251 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:44.294536 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:44.294417 2582 patch_prober.go:28] interesting pod/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 14 01:26:44.294536 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:44.294467 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" podUID="82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 14 01:26:45.294929 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:45.294882 2582 patch_prober.go:28] interesting pod/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 14 01:26:45.295427 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:45.294952 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" podUID="82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 14 01:26:46.294119 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:46.294069 2582 patch_prober.go:28] interesting pod/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 14 01:26:46.294355 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:46.294144 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" podUID="82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 14 01:26:47.294557 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:47.294511 2582 patch_prober.go:28] interesting pod/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 14 01:26:47.295050 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:47.294581 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" podUID="82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 14 01:26:48.298709 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:48.298673 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:48.299067 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:48.298945 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:48.299822 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:48.299802 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:26:56.522851 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:56.522813 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt"] Jul 14 01:26:56.523369 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:26:56.523012 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" podUID="82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" containerName="istio-proxy" containerID="cri-o://284030cc0174f7f75316bfc144eed891c4af048295a09b63f30b6c75d0eed632" gracePeriod=30 Jul 14 01:27:01.784642 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.784567 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:27:01.859624 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.859593 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-envoy\") pod \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " Jul 14 01:27:01.859808 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.859655 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-token\") pod \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " Jul 14 01:27:01.859808 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.859716 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-workload-socket\") pod \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " Jul 14 01:27:01.859808 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.859744 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6psnj\" (UniqueName: \"kubernetes.io/projected/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-kube-api-access-6psnj\") pod \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " Jul 14 01:27:01.859808 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.859783 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-credential-socket\") pod \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " Jul 14 01:27:01.860014 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.859827 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-data\") pod \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " Jul 14 01:27:01.860014 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.859871 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-podinfo\") pod \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " Jul 14 01:27:01.860118 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.860036 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" (UID: "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:27:01.860298 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.860272 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" (UID: "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:27:01.860503 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.860296 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-workload-certs\") pod \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " Jul 14 01:27:01.860676 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.860644 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istiod-ca-cert\") pod \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\" (UID: \"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2\") " Jul 14 01:27:01.860767 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.860495 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" (UID: "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:27:01.860767 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.860685 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-data" (OuterVolumeSpecName: "istio-data") pod "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" (UID: "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:27:01.861005 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.860988 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-workload-certs\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:27:01.861140 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.861129 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-workload-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:27:01.861227 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.861217 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-credential-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:27:01.861314 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.861301 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-data\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:27:01.861447 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.861140 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" (UID: "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:27:01.862661 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.862630 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-token" (OuterVolumeSpecName: "istio-token") pod "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" (UID: "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:27:01.862811 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.862784 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-kube-api-access-6psnj" (OuterVolumeSpecName: "kube-api-access-6psnj") pod "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" (UID: "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2"). InnerVolumeSpecName "kube-api-access-6psnj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:27:01.862811 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.862797 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" (UID: "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:27:01.862966 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.862806 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" (UID: "82c369da-39cc-4c1c-8cd0-a08f8f98c1c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:27:01.962885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.962850 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istiod-ca-cert\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:27:01.962885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.962883 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-envoy\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:27:01.962885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.962892 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-token\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:27:01.963110 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.962900 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6psnj\" (UniqueName: \"kubernetes.io/projected/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-kube-api-access-6psnj\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:27:01.963110 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:01.962910 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2-istio-podinfo\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:27:02.602552 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.602515 2582 generic.go:356] "Generic (PLEG): container finished" podID="82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" containerID="284030cc0174f7f75316bfc144eed891c4af048295a09b63f30b6c75d0eed632" exitCode=0 Jul 14 01:27:02.602846 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.602571 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" Jul 14 01:27:02.602846 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.602608 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" event={"ID":"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2","Type":"ContainerDied","Data":"284030cc0174f7f75316bfc144eed891c4af048295a09b63f30b6c75d0eed632"} Jul 14 01:27:02.602846 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.602678 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt" event={"ID":"82c369da-39cc-4c1c-8cd0-a08f8f98c1c2","Type":"ContainerDied","Data":"b87e1bf9427bee1996d22944f90de4c82bb36ed908d3266bd474414e02806388"} Jul 14 01:27:02.602846 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.602711 2582 scope.go:117] "RemoveContainer" containerID="284030cc0174f7f75316bfc144eed891c4af048295a09b63f30b6c75d0eed632" Jul 14 01:27:02.611739 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.611720 2582 scope.go:117] "RemoveContainer" containerID="6f6be05afdfe2c80d2bfb16c383b0800e662ae5c221a1ebfc6fda5b1c7aa263f" Jul 14 01:27:02.620080 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.620060 2582 scope.go:117] "RemoveContainer" containerID="284030cc0174f7f75316bfc144eed891c4af048295a09b63f30b6c75d0eed632" Jul 14 01:27:02.620351 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:27:02.620311 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"284030cc0174f7f75316bfc144eed891c4af048295a09b63f30b6c75d0eed632\": container with ID starting with 284030cc0174f7f75316bfc144eed891c4af048295a09b63f30b6c75d0eed632 not found: ID does not exist" containerID="284030cc0174f7f75316bfc144eed891c4af048295a09b63f30b6c75d0eed632" Jul 14 01:27:02.620434 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.620364 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"284030cc0174f7f75316bfc144eed891c4af048295a09b63f30b6c75d0eed632"} err="failed to get container status \"284030cc0174f7f75316bfc144eed891c4af048295a09b63f30b6c75d0eed632\": rpc error: code = NotFound desc = could not find container \"284030cc0174f7f75316bfc144eed891c4af048295a09b63f30b6c75d0eed632\": container with ID starting with 284030cc0174f7f75316bfc144eed891c4af048295a09b63f30b6c75d0eed632 not found: ID does not exist" Jul 14 01:27:02.620434 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.620389 2582 scope.go:117] "RemoveContainer" containerID="6f6be05afdfe2c80d2bfb16c383b0800e662ae5c221a1ebfc6fda5b1c7aa263f" Jul 14 01:27:02.620636 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:27:02.620619 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f6be05afdfe2c80d2bfb16c383b0800e662ae5c221a1ebfc6fda5b1c7aa263f\": container with ID starting with 6f6be05afdfe2c80d2bfb16c383b0800e662ae5c221a1ebfc6fda5b1c7aa263f not found: ID does not exist" containerID="6f6be05afdfe2c80d2bfb16c383b0800e662ae5c221a1ebfc6fda5b1c7aa263f" Jul 14 01:27:02.620694 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.620647 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f6be05afdfe2c80d2bfb16c383b0800e662ae5c221a1ebfc6fda5b1c7aa263f"} err="failed to get container status \"6f6be05afdfe2c80d2bfb16c383b0800e662ae5c221a1ebfc6fda5b1c7aa263f\": rpc error: code = NotFound desc = could not find container \"6f6be05afdfe2c80d2bfb16c383b0800e662ae5c221a1ebfc6fda5b1c7aa263f\": container with ID starting with 6f6be05afdfe2c80d2bfb16c383b0800e662ae5c221a1ebfc6fda5b1c7aa263f not found: ID does not exist" Jul 14 01:27:02.623789 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.623767 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt"] Jul 14 01:27:02.627355 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.627315 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c626eb13-openshift-default-77dd58664c-w7gzt"] Jul 14 01:27:02.915052 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.914977 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p"] Jul 14 01:27:02.915412 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.915334 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" containerName="istio-proxy" Jul 14 01:27:02.915412 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.915349 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" containerName="istio-proxy" Jul 14 01:27:02.915412 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.915362 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" containerName="istio-proxy" Jul 14 01:27:02.915412 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.915369 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" containerName="istio-proxy" Jul 14 01:27:02.915412 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.915375 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85fbd4c9-f768-4e18-abc9-186632b6688a" containerName="revoke-keys" Jul 14 01:27:02.915412 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.915381 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="85fbd4c9-f768-4e18-abc9-186632b6688a" containerName="revoke-keys" Jul 14 01:27:02.915412 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.915392 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85fbd4c9-f768-4e18-abc9-186632b6688a" containerName="revoke-keys" Jul 14 01:27:02.915412 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.915398 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="85fbd4c9-f768-4e18-abc9-186632b6688a" containerName="revoke-keys" Jul 14 01:27:02.915646 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.915462 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" containerName="istio-proxy" Jul 14 01:27:02.915646 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.915471 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="85fbd4c9-f768-4e18-abc9-186632b6688a" containerName="revoke-keys" Jul 14 01:27:02.915646 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.915477 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="85fbd4c9-f768-4e18-abc9-186632b6688a" containerName="revoke-keys" Jul 14 01:27:02.915646 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.915486 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" containerName="istio-proxy" Jul 14 01:27:02.919824 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.919798 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p" Jul 14 01:27:02.922458 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.922434 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zn7l8\"" Jul 14 01:27:02.922595 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.922434 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 01:27:02.925199 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:02.925169 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p"] Jul 14 01:27:03.072607 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:03.072563 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85bc53df-89d0-4368-8a85-1c7daacd4d25-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4928854b-bv68p\" (UID: \"85bc53df-89d0-4368-8a85-1c7daacd4d25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p" Jul 14 01:27:03.173984 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:03.173872 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85bc53df-89d0-4368-8a85-1c7daacd4d25-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4928854b-bv68p\" (UID: \"85bc53df-89d0-4368-8a85-1c7daacd4d25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p" Jul 14 01:27:03.174629 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:03.174605 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85bc53df-89d0-4368-8a85-1c7daacd4d25-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4928854b-bv68p\" (UID: \"85bc53df-89d0-4368-8a85-1c7daacd4d25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p" Jul 14 01:27:03.235850 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:03.235816 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p" Jul 14 01:27:03.373021 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:03.372996 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p"] Jul 14 01:27:03.374899 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:27:03.374871 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85bc53df_89d0_4368_8a85_1c7daacd4d25.slice/crio-86b865570c280f69055162ad22ba1c3e66fe2b3a5772335834ce29211c3760ca WatchSource:0}: Error finding container 86b865570c280f69055162ad22ba1c3e66fe2b3a5772335834ce29211c3760ca: Status 404 returned error can't find the container with id 86b865570c280f69055162ad22ba1c3e66fe2b3a5772335834ce29211c3760ca Jul 14 01:27:03.609223 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:03.609188 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p" event={"ID":"85bc53df-89d0-4368-8a85-1c7daacd4d25","Type":"ContainerStarted","Data":"86b865570c280f69055162ad22ba1c3e66fe2b3a5772335834ce29211c3760ca"} Jul 14 01:27:03.694067 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:03.694039 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82c369da-39cc-4c1c-8cd0-a08f8f98c1c2" path="/var/lib/kubelet/pods/82c369da-39cc-4c1c-8cd0-a08f8f98c1c2/volumes" Jul 14 01:27:04.616960 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:04.616919 2582 generic.go:356] "Generic (PLEG): container finished" podID="85bc53df-89d0-4368-8a85-1c7daacd4d25" containerID="8a6dfb8aa73bfe2bb8b953f76ada8cf893ad43c1042d3b531ce1669ddaedf5cd" exitCode=7 Jul 14 01:27:04.617474 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:04.617003 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p" event={"ID":"85bc53df-89d0-4368-8a85-1c7daacd4d25","Type":"ContainerDied","Data":"8a6dfb8aa73bfe2bb8b953f76ada8cf893ad43c1042d3b531ce1669ddaedf5cd"} Jul 14 01:27:04.617474 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:04.617198 2582 scope.go:117] "RemoveContainer" containerID="8a6dfb8aa73bfe2bb8b953f76ada8cf893ad43c1042d3b531ce1669ddaedf5cd" Jul 14 01:27:05.624129 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:05.624098 2582 generic.go:356] "Generic (PLEG): container finished" podID="85bc53df-89d0-4368-8a85-1c7daacd4d25" containerID="a66eb9cf06588db3f19dbdf38fdbde2d81d4fd780a1c94d6aa7c9aa0ba1b9717" exitCode=7 Jul 14 01:27:05.624594 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:05.624156 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p" event={"ID":"85bc53df-89d0-4368-8a85-1c7daacd4d25","Type":"ContainerDied","Data":"a66eb9cf06588db3f19dbdf38fdbde2d81d4fd780a1c94d6aa7c9aa0ba1b9717"} Jul 14 01:27:05.624594 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:05.624203 2582 scope.go:117] "RemoveContainer" containerID="8a6dfb8aa73bfe2bb8b953f76ada8cf893ad43c1042d3b531ce1669ddaedf5cd" Jul 14 01:27:05.624594 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:05.624440 2582 scope.go:117] "RemoveContainer" containerID="a66eb9cf06588db3f19dbdf38fdbde2d81d4fd780a1c94d6aa7c9aa0ba1b9717" Jul 14 01:27:05.624701 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:27:05.624636 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-4928854b-bv68p_odh-ai-gateway-infra(85bc53df-89d0-4368-8a85-1c7daacd4d25)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p" podUID="85bc53df-89d0-4368-8a85-1c7daacd4d25" Jul 14 01:27:06.631239 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:06.631212 2582 scope.go:117] "RemoveContainer" containerID="a66eb9cf06588db3f19dbdf38fdbde2d81d4fd780a1c94d6aa7c9aa0ba1b9717" Jul 14 01:27:06.631653 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:27:06.631441 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-4928854b-bv68p_odh-ai-gateway-infra(85bc53df-89d0-4368-8a85-1c7daacd4d25)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p" podUID="85bc53df-89d0-4368-8a85-1c7daacd4d25" Jul 14 01:27:21.690768 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:21.690735 2582 scope.go:117] "RemoveContainer" containerID="a66eb9cf06588db3f19dbdf38fdbde2d81d4fd780a1c94d6aa7c9aa0ba1b9717" Jul 14 01:27:22.710422 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:22.710383 2582 generic.go:356] "Generic (PLEG): container finished" podID="85bc53df-89d0-4368-8a85-1c7daacd4d25" containerID="b49bab9e6e7cd74411cf26118f17b1bdacc970f348997f36056e54129ea315b8" exitCode=0 Jul 14 01:27:22.710816 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:22.710454 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p" event={"ID":"85bc53df-89d0-4368-8a85-1c7daacd4d25","Type":"ContainerDied","Data":"b49bab9e6e7cd74411cf26118f17b1bdacc970f348997f36056e54129ea315b8"} Jul 14 01:27:22.710816 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:22.710531 2582 scope.go:117] "RemoveContainer" containerID="a66eb9cf06588db3f19dbdf38fdbde2d81d4fd780a1c94d6aa7c9aa0ba1b9717" Jul 14 01:27:23.737803 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:23.737772 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p"] Jul 14 01:27:23.855914 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:23.855891 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p" Jul 14 01:27:23.963235 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:23.963204 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85bc53df-89d0-4368-8a85-1c7daacd4d25-maas-api-service-ca\") pod \"85bc53df-89d0-4368-8a85-1c7daacd4d25\" (UID: \"85bc53df-89d0-4368-8a85-1c7daacd4d25\") " Jul 14 01:27:23.963559 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:23.963537 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85bc53df-89d0-4368-8a85-1c7daacd4d25-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "85bc53df-89d0-4368-8a85-1c7daacd4d25" (UID: "85bc53df-89d0-4368-8a85-1c7daacd4d25"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:27:24.064007 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:24.063911 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85bc53df-89d0-4368-8a85-1c7daacd4d25-maas-api-service-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:27:24.722388 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:24.722358 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p" Jul 14 01:27:24.722559 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:24.722384 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p" event={"ID":"85bc53df-89d0-4368-8a85-1c7daacd4d25","Type":"ContainerDied","Data":"86b865570c280f69055162ad22ba1c3e66fe2b3a5772335834ce29211c3760ca"} Jul 14 01:27:24.722559 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:24.722434 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86b865570c280f69055162ad22ba1c3e66fe2b3a5772335834ce29211c3760ca" Jul 14 01:27:24.750754 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:24.750716 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p"] Jul 14 01:27:24.754371 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:24.754345 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-bv68p"] Jul 14 01:27:25.692959 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:27:25.692918 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85bc53df-89d0-4368-8a85-1c7daacd4d25" path="/var/lib/kubelet/pods/85bc53df-89d0-4368-8a85-1c7daacd4d25/volumes" Jul 14 01:28:02.961139 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:02.961059 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv"] Jul 14 01:28:02.961577 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:02.961438 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85bc53df-89d0-4368-8a85-1c7daacd4d25" containerName="revoke-keys" Jul 14 01:28:02.961577 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:02.961451 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="85bc53df-89d0-4368-8a85-1c7daacd4d25" containerName="revoke-keys" Jul 14 01:28:02.961577 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:02.961469 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85bc53df-89d0-4368-8a85-1c7daacd4d25" containerName="revoke-keys" Jul 14 01:28:02.961577 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:02.961475 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="85bc53df-89d0-4368-8a85-1c7daacd4d25" containerName="revoke-keys" Jul 14 01:28:02.961577 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:02.961529 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="85bc53df-89d0-4368-8a85-1c7daacd4d25" containerName="revoke-keys" Jul 14 01:28:02.961577 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:02.961540 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="85bc53df-89d0-4368-8a85-1c7daacd4d25" containerName="revoke-keys" Jul 14 01:28:02.964365 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:02.964349 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv" Jul 14 01:28:02.966824 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:02.966807 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 01:28:02.966905 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:02.966817 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zn7l8\"" Jul 14 01:28:02.971053 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:02.971030 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv"] Jul 14 01:28:03.017393 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:03.017348 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/414c797b-d977-4668-b1f1-0e7bd3cd06ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv\" (UID: \"414c797b-d977-4668-b1f1-0e7bd3cd06ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv" Jul 14 01:28:03.117921 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:03.117875 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/414c797b-d977-4668-b1f1-0e7bd3cd06ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv\" (UID: \"414c797b-d977-4668-b1f1-0e7bd3cd06ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv" Jul 14 01:28:03.118621 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:03.118597 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/414c797b-d977-4668-b1f1-0e7bd3cd06ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv\" (UID: \"414c797b-d977-4668-b1f1-0e7bd3cd06ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv" Jul 14 01:28:03.275907 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:03.275820 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv" Jul 14 01:28:03.416007 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:03.415971 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv"] Jul 14 01:28:03.418998 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:28:03.418969 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod414c797b_d977_4668_b1f1_0e7bd3cd06ff.slice/crio-a6e0b74bd7af069c21dac22d2ace1cc1e98a0f3739baa9095ff775ae2d439a7e WatchSource:0}: Error finding container a6e0b74bd7af069c21dac22d2ace1cc1e98a0f3739baa9095ff775ae2d439a7e: Status 404 returned error can't find the container with id a6e0b74bd7af069c21dac22d2ace1cc1e98a0f3739baa9095ff775ae2d439a7e Jul 14 01:28:03.913238 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:03.913140 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv" event={"ID":"414c797b-d977-4668-b1f1-0e7bd3cd06ff","Type":"ContainerStarted","Data":"3a365e0da9d1fcb4f077f6226336eba10fac6bfe141d6d63d47048c635e94bb2"} Jul 14 01:28:03.913238 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:03.913197 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv" event={"ID":"414c797b-d977-4668-b1f1-0e7bd3cd06ff","Type":"ContainerStarted","Data":"a6e0b74bd7af069c21dac22d2ace1cc1e98a0f3739baa9095ff775ae2d439a7e"} Jul 14 01:28:03.929641 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:03.929595 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv" podStartSLOduration=1.929581088 podStartE2EDuration="1.929581088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:28:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:28:03.926990999 +0000 UTC m=+1712.829846500" watchObservedRunningTime="2026-07-14 01:28:03.929581088 +0000 UTC m=+1712.832436587" Jul 14 01:28:04.919257 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:04.919220 2582 generic.go:356] "Generic (PLEG): container finished" podID="414c797b-d977-4668-b1f1-0e7bd3cd06ff" containerID="3a365e0da9d1fcb4f077f6226336eba10fac6bfe141d6d63d47048c635e94bb2" exitCode=0 Jul 14 01:28:04.919657 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:04.919307 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv" event={"ID":"414c797b-d977-4668-b1f1-0e7bd3cd06ff","Type":"ContainerDied","Data":"3a365e0da9d1fcb4f077f6226336eba10fac6bfe141d6d63d47048c635e94bb2"} Jul 14 01:28:06.058240 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:06.058213 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv" Jul 14 01:28:06.146591 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:06.146550 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/414c797b-d977-4668-b1f1-0e7bd3cd06ff-maas-api-service-ca\") pod \"414c797b-d977-4668-b1f1-0e7bd3cd06ff\" (UID: \"414c797b-d977-4668-b1f1-0e7bd3cd06ff\") " Jul 14 01:28:06.146956 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:06.146933 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/414c797b-d977-4668-b1f1-0e7bd3cd06ff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "414c797b-d977-4668-b1f1-0e7bd3cd06ff" (UID: "414c797b-d977-4668-b1f1-0e7bd3cd06ff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:28:06.248353 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:06.248244 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/414c797b-d977-4668-b1f1-0e7bd3cd06ff-maas-api-service-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:28:06.930799 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:06.930768 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv" Jul 14 01:28:06.930977 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:06.930761 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv" event={"ID":"414c797b-d977-4668-b1f1-0e7bd3cd06ff","Type":"ContainerDied","Data":"a6e0b74bd7af069c21dac22d2ace1cc1e98a0f3739baa9095ff775ae2d439a7e"} Jul 14 01:28:06.930977 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:06.930892 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6e0b74bd7af069c21dac22d2ace1cc1e98a0f3739baa9095ff775ae2d439a7e" Jul 14 01:28:14.732863 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.732823 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls"] Jul 14 01:28:14.733856 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.733826 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="414c797b-d977-4668-b1f1-0e7bd3cd06ff" containerName="revoke-keys" Jul 14 01:28:14.733960 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.733870 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="414c797b-d977-4668-b1f1-0e7bd3cd06ff" containerName="revoke-keys" Jul 14 01:28:14.733960 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.733901 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85bc53df-89d0-4368-8a85-1c7daacd4d25" containerName="revoke-keys" Jul 14 01:28:14.733960 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.733910 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="85bc53df-89d0-4368-8a85-1c7daacd4d25" containerName="revoke-keys" Jul 14 01:28:14.734098 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.734038 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="85bc53df-89d0-4368-8a85-1c7daacd4d25" containerName="revoke-keys" Jul 14 01:28:14.734098 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.734049 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="414c797b-d977-4668-b1f1-0e7bd3cd06ff" containerName="revoke-keys" Jul 14 01:28:14.737809 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.737782 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.740592 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.740565 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e1db3cf4-openshift-default-dockercfg-thvn2\"" Jul 14 01:28:14.748586 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.748560 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls"] Jul 14 01:28:14.824311 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.824271 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-workload-socket\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.824496 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.824343 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-data\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.824496 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.824364 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-credential-socket\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.824496 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.824382 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-workload-certs\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.824496 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.824446 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-token\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.824496 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.824481 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istiod-ca-cert\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.824699 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.824571 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-envoy\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.824699 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.824621 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-podinfo\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.824699 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.824637 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc5r5\" (UniqueName: \"kubernetes.io/projected/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-kube-api-access-tc5r5\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.925616 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.925575 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-envoy\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.925803 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.925637 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-podinfo\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.925803 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.925660 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc5r5\" (UniqueName: \"kubernetes.io/projected/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-kube-api-access-tc5r5\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.925803 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.925689 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-workload-socket\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.925803 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.925721 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-data\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.926009 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.925866 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-credential-socket\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.926009 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.925914 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-workload-certs\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.926009 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.925970 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-token\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.926009 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.925994 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istiod-ca-cert\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.926210 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.926105 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-data\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.926210 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.926144 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-workload-socket\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.926210 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.926193 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-workload-certs\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.926399 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.926378 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-credential-socket\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.926984 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.926960 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istiod-ca-cert\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.928169 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.928139 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-envoy\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.928308 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.928292 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-podinfo\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.933710 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.933688 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-token\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:14.933820 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:14.933771 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc5r5\" (UniqueName: \"kubernetes.io/projected/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-kube-api-access-tc5r5\") pod \"e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:15.060364 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:15.060242 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:15.203787 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:15.203757 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls"] Jul 14 01:28:15.206481 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:28:15.206447 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f9cde23_6f1e_47c4_88cb_12df2fc5c590.slice/crio-69247f3b21e2f77ff29d29112787f12d3111d0a3e56320cd91f2fdab8df235ea WatchSource:0}: Error finding container 69247f3b21e2f77ff29d29112787f12d3111d0a3e56320cd91f2fdab8df235ea: Status 404 returned error can't find the container with id 69247f3b21e2f77ff29d29112787f12d3111d0a3e56320cd91f2fdab8df235ea Jul 14 01:28:15.208565 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:15.208526 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:28:15.208655 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:15.208616 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:28:15.208695 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:15.208660 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:28:15.978472 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:15.978429 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" event={"ID":"1f9cde23-6f1e-47c4-88cb-12df2fc5c590","Type":"ContainerStarted","Data":"ebf150596054725c13dd8c95f3efe3372815a7a32c4dab730f5133b36fa59720"} Jul 14 01:28:15.978997 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:15.978496 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" event={"ID":"1f9cde23-6f1e-47c4-88cb-12df2fc5c590","Type":"ContainerStarted","Data":"69247f3b21e2f77ff29d29112787f12d3111d0a3e56320cd91f2fdab8df235ea"} Jul 14 01:28:16.060901 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:16.060875 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:16.062413 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:16.062352 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" podStartSLOduration=2.062308493 podStartE2EDuration="2.062308493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:28:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:28:16.060026883 +0000 UTC m=+1724.962882382" watchObservedRunningTime="2026-07-14 01:28:16.062308493 +0000 UTC m=+1724.965163993" Jul 14 01:28:16.066557 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:16.066532 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:16.983733 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:16.983699 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:16.984574 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:16.984554 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:21.041521 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:21.041490 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5"] Jul 14 01:28:21.045272 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:21.045245 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" Jul 14 01:28:21.047585 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:21.047555 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zn7l8\"" Jul 14 01:28:21.047923 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:21.047906 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 01:28:21.052272 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:21.052244 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5"] Jul 14 01:28:21.192173 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:21.192135 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c83fa315-d766-4bd8-836b-fd85095592cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5\" (UID: \"c83fa315-d766-4bd8-836b-fd85095592cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" Jul 14 01:28:21.293524 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:21.293418 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c83fa315-d766-4bd8-836b-fd85095592cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5\" (UID: \"c83fa315-d766-4bd8-836b-fd85095592cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" Jul 14 01:28:21.294625 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:21.294541 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c83fa315-d766-4bd8-836b-fd85095592cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5\" (UID: \"c83fa315-d766-4bd8-836b-fd85095592cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" Jul 14 01:28:21.357166 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:21.357120 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" Jul 14 01:28:21.548916 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:21.548879 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5"] Jul 14 01:28:21.551446 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:28:21.551416 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc83fa315_d766_4bd8_836b_fd85095592cc.slice/crio-45b6319e4fcb1b37883087a90c0c1637d1fa09a51c8c2f1ca7c1fba80017fda9 WatchSource:0}: Error finding container 45b6319e4fcb1b37883087a90c0c1637d1fa09a51c8c2f1ca7c1fba80017fda9: Status 404 returned error can't find the container with id 45b6319e4fcb1b37883087a90c0c1637d1fa09a51c8c2f1ca7c1fba80017fda9 Jul 14 01:28:22.016889 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:22.016854 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" event={"ID":"c83fa315-d766-4bd8-836b-fd85095592cc","Type":"ContainerStarted","Data":"1c0769b692f5da53860adb1db23fd03e8af99db6e88d17550f3d3c847f88dc75"} Jul 14 01:28:22.017035 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:22.016905 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" event={"ID":"c83fa315-d766-4bd8-836b-fd85095592cc","Type":"ContainerStarted","Data":"45b6319e4fcb1b37883087a90c0c1637d1fa09a51c8c2f1ca7c1fba80017fda9"} Jul 14 01:28:22.032058 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:22.032002 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" podStartSLOduration=1.031986135 podStartE2EDuration="1.031986135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:28:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:28:22.030671795 +0000 UTC m=+1730.933527295" watchObservedRunningTime="2026-07-14 01:28:22.031986135 +0000 UTC m=+1730.934841634" Jul 14 01:28:23.023268 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:23.023235 2582 generic.go:356] "Generic (PLEG): container finished" podID="c83fa315-d766-4bd8-836b-fd85095592cc" containerID="1c0769b692f5da53860adb1db23fd03e8af99db6e88d17550f3d3c847f88dc75" exitCode=7 Jul 14 01:28:23.023710 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:23.023352 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" event={"ID":"c83fa315-d766-4bd8-836b-fd85095592cc","Type":"ContainerDied","Data":"1c0769b692f5da53860adb1db23fd03e8af99db6e88d17550f3d3c847f88dc75"} Jul 14 01:28:23.023710 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:23.023546 2582 scope.go:117] "RemoveContainer" containerID="1c0769b692f5da53860adb1db23fd03e8af99db6e88d17550f3d3c847f88dc75" Jul 14 01:28:24.030307 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:24.030274 2582 generic.go:356] "Generic (PLEG): container finished" podID="c83fa315-d766-4bd8-836b-fd85095592cc" containerID="ced0413688e12f92656a901681c2f3e6edc352b43560ee46a73bb1c736843566" exitCode=7 Jul 14 01:28:24.030696 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:24.030366 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" event={"ID":"c83fa315-d766-4bd8-836b-fd85095592cc","Type":"ContainerDied","Data":"ced0413688e12f92656a901681c2f3e6edc352b43560ee46a73bb1c736843566"} Jul 14 01:28:24.030696 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:24.030429 2582 scope.go:117] "RemoveContainer" containerID="1c0769b692f5da53860adb1db23fd03e8af99db6e88d17550f3d3c847f88dc75" Jul 14 01:28:24.030696 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:24.030676 2582 scope.go:117] "RemoveContainer" containerID="ced0413688e12f92656a901681c2f3e6edc352b43560ee46a73bb1c736843566" Jul 14 01:28:24.030911 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:28:24.030894 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5_odh-ai-gateway-infra(c83fa315-d766-4bd8-836b-fd85095592cc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" podUID="c83fa315-d766-4bd8-836b-fd85095592cc" Jul 14 01:28:25.038302 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:25.038272 2582 scope.go:117] "RemoveContainer" containerID="ced0413688e12f92656a901681c2f3e6edc352b43560ee46a73bb1c736843566" Jul 14 01:28:25.038801 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:28:25.038529 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5_odh-ai-gateway-infra(c83fa315-d766-4bd8-836b-fd85095592cc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" podUID="c83fa315-d766-4bd8-836b-fd85095592cc" Jul 14 01:28:28.055701 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:28.055667 2582 generic.go:356] "Generic (PLEG): container finished" podID="1f9cde23-6f1e-47c4-88cb-12df2fc5c590" containerID="ebf150596054725c13dd8c95f3efe3372815a7a32c4dab730f5133b36fa59720" exitCode=0 Jul 14 01:28:28.056111 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:28.055719 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" event={"ID":"1f9cde23-6f1e-47c4-88cb-12df2fc5c590","Type":"ContainerDied","Data":"ebf150596054725c13dd8c95f3efe3372815a7a32c4dab730f5133b36fa59720"} Jul 14 01:28:28.056111 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:28.056083 2582 scope.go:117] "RemoveContainer" containerID="ebf150596054725c13dd8c95f3efe3372815a7a32c4dab730f5133b36fa59720" Jul 14 01:28:28.057052 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:28.057025 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:28:28.057102 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:28.057089 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:28:28.057136 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:28.057120 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:28:29.063010 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:29.062975 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" event={"ID":"1f9cde23-6f1e-47c4-88cb-12df2fc5c590","Type":"ContainerStarted","Data":"9e089b9b3e949c6417f06b702423465eb230a11a51cf7476c74e3f7453288550"} Jul 14 01:28:30.061366 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:30.061311 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:30.062421 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:30.062394 2582 patch_prober.go:28] interesting pod/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 14 01:28:30.062525 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:30.062458 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" podUID="1f9cde23-6f1e-47c4-88cb-12df2fc5c590" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 14 01:28:31.061024 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:31.060984 2582 patch_prober.go:28] interesting pod/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 14 01:28:31.061460 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:31.061045 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" podUID="1f9cde23-6f1e-47c4-88cb-12df2fc5c590" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 14 01:28:31.985133 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:31.985083 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:32.060650 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:32.060619 2582 patch_prober.go:28] interesting pod/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 14 01:28:32.060820 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:32.060675 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" podUID="1f9cde23-6f1e-47c4-88cb-12df2fc5c590" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 14 01:28:33.064654 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:33.064621 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:33.065638 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:33.065621 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:28:38.687317 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:38.687285 2582 scope.go:117] "RemoveContainer" containerID="ced0413688e12f92656a901681c2f3e6edc352b43560ee46a73bb1c736843566" Jul 14 01:28:39.118017 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:39.117981 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" event={"ID":"c83fa315-d766-4bd8-836b-fd85095592cc","Type":"ContainerStarted","Data":"8e3cb7b5f14fe2977fb72133659df71bad254dbc5536018043173e5df99b3ddc"} Jul 14 01:28:40.125428 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:40.125287 2582 generic.go:356] "Generic (PLEG): container finished" podID="c83fa315-d766-4bd8-836b-fd85095592cc" containerID="8e3cb7b5f14fe2977fb72133659df71bad254dbc5536018043173e5df99b3ddc" exitCode=0 Jul 14 01:28:40.125428 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:40.125370 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" event={"ID":"c83fa315-d766-4bd8-836b-fd85095592cc","Type":"ContainerDied","Data":"8e3cb7b5f14fe2977fb72133659df71bad254dbc5536018043173e5df99b3ddc"} Jul 14 01:28:40.125428 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:40.125436 2582 scope.go:117] "RemoveContainer" containerID="ced0413688e12f92656a901681c2f3e6edc352b43560ee46a73bb1c736843566" Jul 14 01:28:40.142720 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:40.142694 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5"] Jul 14 01:28:41.265577 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:41.265545 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" Jul 14 01:28:41.282428 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:41.282393 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c83fa315-d766-4bd8-836b-fd85095592cc-maas-api-service-ca\") pod \"c83fa315-d766-4bd8-836b-fd85095592cc\" (UID: \"c83fa315-d766-4bd8-836b-fd85095592cc\") " Jul 14 01:28:41.282843 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:41.282803 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c83fa315-d766-4bd8-836b-fd85095592cc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c83fa315-d766-4bd8-836b-fd85095592cc" (UID: "c83fa315-d766-4bd8-836b-fd85095592cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:28:41.384034 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:41.383939 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c83fa315-d766-4bd8-836b-fd85095592cc-maas-api-service-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:28:42.138658 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:42.138628 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" Jul 14 01:28:42.138830 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:42.138658 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5" event={"ID":"c83fa315-d766-4bd8-836b-fd85095592cc","Type":"ContainerDied","Data":"45b6319e4fcb1b37883087a90c0c1637d1fa09a51c8c2f1ca7c1fba80017fda9"} Jul 14 01:28:42.138830 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:42.138703 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45b6319e4fcb1b37883087a90c0c1637d1fa09a51c8c2f1ca7c1fba80017fda9" Jul 14 01:28:42.167921 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:42.167882 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5"] Jul 14 01:28:42.169787 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:42.169762 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-9rvb5"] Jul 14 01:28:43.692700 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:28:43.692668 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c83fa315-d766-4bd8-836b-fd85095592cc" path="/var/lib/kubelet/pods/c83fa315-d766-4bd8-836b-fd85095592cc/volumes" Jul 14 01:29:21.087929 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.087893 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg"] Jul 14 01:29:21.088408 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.088237 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c83fa315-d766-4bd8-836b-fd85095592cc" containerName="revoke-keys" Jul 14 01:29:21.088408 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.088247 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c83fa315-d766-4bd8-836b-fd85095592cc" containerName="revoke-keys" Jul 14 01:29:21.088408 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.088263 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c83fa315-d766-4bd8-836b-fd85095592cc" containerName="revoke-keys" Jul 14 01:29:21.088408 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.088269 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c83fa315-d766-4bd8-836b-fd85095592cc" containerName="revoke-keys" Jul 14 01:29:21.088408 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.088337 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c83fa315-d766-4bd8-836b-fd85095592cc" containerName="revoke-keys" Jul 14 01:29:21.088408 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.088346 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c83fa315-d766-4bd8-836b-fd85095592cc" containerName="revoke-keys" Jul 14 01:29:21.091421 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.091403 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg" Jul 14 01:29:21.093972 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.093949 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 01:29:21.094096 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.094001 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zn7l8\"" Jul 14 01:29:21.097952 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.097914 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg"] Jul 14 01:29:21.158021 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.157978 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07cc313-904d-49ba-9ddd-4cd5167691d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg\" (UID: \"a07cc313-904d-49ba-9ddd-4cd5167691d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg" Jul 14 01:29:21.258758 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.258724 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07cc313-904d-49ba-9ddd-4cd5167691d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg\" (UID: \"a07cc313-904d-49ba-9ddd-4cd5167691d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg" Jul 14 01:29:21.259395 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.259378 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07cc313-904d-49ba-9ddd-4cd5167691d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg\" (UID: \"a07cc313-904d-49ba-9ddd-4cd5167691d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg" Jul 14 01:29:21.403077 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.402986 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg" Jul 14 01:29:21.537154 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:21.537121 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg"] Jul 14 01:29:21.539261 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:29:21.539228 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda07cc313_904d_49ba_9ddd_4cd5167691d2.slice/crio-e7122436ebc7e44a888a75b0b60aaca82ca06755545ce0a35f1c2d3abd30848d WatchSource:0}: Error finding container e7122436ebc7e44a888a75b0b60aaca82ca06755545ce0a35f1c2d3abd30848d: Status 404 returned error can't find the container with id e7122436ebc7e44a888a75b0b60aaca82ca06755545ce0a35f1c2d3abd30848d Jul 14 01:29:22.342368 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:22.342315 2582 generic.go:356] "Generic (PLEG): container finished" podID="a07cc313-904d-49ba-9ddd-4cd5167691d2" containerID="face7e07e09170a219dbb5c0a41e76e091919c3ff31e3fa57dfef2ebd26c6183" exitCode=0 Jul 14 01:29:22.342762 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:22.342408 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg" event={"ID":"a07cc313-904d-49ba-9ddd-4cd5167691d2","Type":"ContainerDied","Data":"face7e07e09170a219dbb5c0a41e76e091919c3ff31e3fa57dfef2ebd26c6183"} Jul 14 01:29:22.342762 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:22.342445 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg" event={"ID":"a07cc313-904d-49ba-9ddd-4cd5167691d2","Type":"ContainerStarted","Data":"e7122436ebc7e44a888a75b0b60aaca82ca06755545ce0a35f1c2d3abd30848d"} Jul 14 01:29:23.476894 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:23.476868 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg" Jul 14 01:29:23.576251 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:23.576219 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07cc313-904d-49ba-9ddd-4cd5167691d2-maas-api-service-ca\") pod \"a07cc313-904d-49ba-9ddd-4cd5167691d2\" (UID: \"a07cc313-904d-49ba-9ddd-4cd5167691d2\") " Jul 14 01:29:23.576592 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:23.576571 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a07cc313-904d-49ba-9ddd-4cd5167691d2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a07cc313-904d-49ba-9ddd-4cd5167691d2" (UID: "a07cc313-904d-49ba-9ddd-4cd5167691d2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:29:23.677951 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:23.677852 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07cc313-904d-49ba-9ddd-4cd5167691d2-maas-api-service-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:29:24.353371 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:24.353338 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg" Jul 14 01:29:24.353371 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:24.353357 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg" event={"ID":"a07cc313-904d-49ba-9ddd-4cd5167691d2","Type":"ContainerDied","Data":"e7122436ebc7e44a888a75b0b60aaca82ca06755545ce0a35f1c2d3abd30848d"} Jul 14 01:29:24.353569 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:24.353436 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7122436ebc7e44a888a75b0b60aaca82ca06755545ce0a35f1c2d3abd30848d" Jul 14 01:29:31.694690 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:31.694657 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:29:31.697282 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:31.697246 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:29:47.258226 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.258190 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls"] Jul 14 01:29:47.258634 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.258404 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" podUID="1f9cde23-6f1e-47c4-88cb-12df2fc5c590" containerName="istio-proxy" containerID="cri-o://9e089b9b3e949c6417f06b702423465eb230a11a51cf7476c74e3f7453288550" gracePeriod=30 Jul 14 01:29:47.553423 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.553339 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5"] Jul 14 01:29:47.553886 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.553864 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a07cc313-904d-49ba-9ddd-4cd5167691d2" containerName="revoke-keys" Jul 14 01:29:47.553933 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.553891 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a07cc313-904d-49ba-9ddd-4cd5167691d2" containerName="revoke-keys" Jul 14 01:29:47.553933 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.553916 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c83fa315-d766-4bd8-836b-fd85095592cc" containerName="revoke-keys" Jul 14 01:29:47.553933 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.553925 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c83fa315-d766-4bd8-836b-fd85095592cc" containerName="revoke-keys" Jul 14 01:29:47.554028 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.554017 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a07cc313-904d-49ba-9ddd-4cd5167691d2" containerName="revoke-keys" Jul 14 01:29:47.554063 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.554029 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c83fa315-d766-4bd8-836b-fd85095592cc" containerName="revoke-keys" Jul 14 01:29:47.557407 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.557387 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.559920 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.559895 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-d3208c92-openshift-default-dockercfg-dc8tt\"" Jul 14 01:29:47.567314 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.567289 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5"] Jul 14 01:29:47.695467 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.695440 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-podinfo\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.695467 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.695472 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd22z\" (UniqueName: \"kubernetes.io/projected/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-kube-api-access-bd22z\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.695691 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.695492 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-data\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.695691 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.695518 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-token\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.695691 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.695637 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-credential-socket\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.695691 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.695667 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-workload-certs\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.695691 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.695682 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istiod-ca-cert\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.695857 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.695701 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-workload-socket\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.695857 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.695753 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-envoy\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.797091 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.797050 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-podinfo\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.797091 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.797090 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd22z\" (UniqueName: \"kubernetes.io/projected/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-kube-api-access-bd22z\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.797375 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.797123 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-data\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.797375 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.797156 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-token\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.797375 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.797346 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-credential-socket\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.797540 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.797396 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-workload-certs\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.797540 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.797422 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istiod-ca-cert\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.797540 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.797453 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-workload-socket\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.797540 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.797501 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-envoy\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.797873 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.797847 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-credential-socket\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.797990 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.797973 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-data\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.798058 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.798003 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-workload-certs\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.798058 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.797920 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-workload-socket\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.798509 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.798486 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istiod-ca-cert\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.799824 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.799797 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-envoy\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.799931 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.799915 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-podinfo\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.805029 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.804962 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-token\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.805250 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.805232 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd22z\" (UniqueName: \"kubernetes.io/projected/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-kube-api-access-bd22z\") pod \"e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:47.870292 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:47.870259 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:48.013876 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:48.013846 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5"] Jul 14 01:29:48.015283 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:29:48.015256 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9afbdbda_990a_4c4c_adfc_4018f5fd1a7e.slice/crio-0022e256e861bc14157d44854ba2e56cca6906599202fd4f20f07affbb103752 WatchSource:0}: Error finding container 0022e256e861bc14157d44854ba2e56cca6906599202fd4f20f07affbb103752: Status 404 returned error can't find the container with id 0022e256e861bc14157d44854ba2e56cca6906599202fd4f20f07affbb103752 Jul 14 01:29:48.017578 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:48.017545 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:29:48.017657 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:48.017616 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:29:48.017695 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:48.017653 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:29:48.067065 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:48.067025 2582 patch_prober.go:28] interesting pod/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 01:29:48.067190 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:48.067095 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" podUID="1f9cde23-6f1e-47c4-88cb-12df2fc5c590" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 01:29:48.483847 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:48.483811 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" event={"ID":"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e","Type":"ContainerStarted","Data":"6aaf60f3731c9cfb98128bc9cf0de6a694fbe10408a4364d8c873a2944f2f980"} Jul 14 01:29:48.484349 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:48.483897 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" event={"ID":"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e","Type":"ContainerStarted","Data":"0022e256e861bc14157d44854ba2e56cca6906599202fd4f20f07affbb103752"} Jul 14 01:29:48.506762 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:48.506693 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" podStartSLOduration=1.5066736600000001 podStartE2EDuration="1.50667366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:29:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:29:48.503142632 +0000 UTC m=+1817.405998134" watchObservedRunningTime="2026-07-14 01:29:48.50667366 +0000 UTC m=+1817.409529160" Jul 14 01:29:48.871410 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:48.871152 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:49.876668 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:49.876633 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:50.495508 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:50.495475 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:50.496440 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:50.496420 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:29:52.513890 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.513860 2582 generic.go:356] "Generic (PLEG): container finished" podID="1f9cde23-6f1e-47c4-88cb-12df2fc5c590" containerID="9e089b9b3e949c6417f06b702423465eb230a11a51cf7476c74e3f7453288550" exitCode=0 Jul 14 01:29:52.514242 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.514130 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" event={"ID":"1f9cde23-6f1e-47c4-88cb-12df2fc5c590","Type":"ContainerDied","Data":"9e089b9b3e949c6417f06b702423465eb230a11a51cf7476c74e3f7453288550"} Jul 14 01:29:52.514242 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.514201 2582 scope.go:117] "RemoveContainer" containerID="ebf150596054725c13dd8c95f3efe3372815a7a32c4dab730f5133b36fa59720" Jul 14 01:29:52.559703 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.559677 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:29:52.642817 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.642786 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-envoy\") pod \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " Jul 14 01:29:52.642817 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.642832 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-workload-socket\") pod \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " Jul 14 01:29:52.643055 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.642859 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-token\") pod \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " Jul 14 01:29:52.643055 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.642916 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-podinfo\") pod \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " Jul 14 01:29:52.643055 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.642950 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-workload-certs\") pod \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " Jul 14 01:29:52.643055 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.642971 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istiod-ca-cert\") pod \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " Jul 14 01:29:52.643055 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.643024 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-data\") pod \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " Jul 14 01:29:52.643055 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.643048 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tc5r5\" (UniqueName: \"kubernetes.io/projected/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-kube-api-access-tc5r5\") pod \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " Jul 14 01:29:52.643385 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.643086 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-credential-socket\") pod \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\" (UID: \"1f9cde23-6f1e-47c4-88cb-12df2fc5c590\") " Jul 14 01:29:52.643385 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.643111 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1f9cde23-6f1e-47c4-88cb-12df2fc5c590" (UID: "1f9cde23-6f1e-47c4-88cb-12df2fc5c590"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:29:52.643491 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.643440 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1f9cde23-6f1e-47c4-88cb-12df2fc5c590" (UID: "1f9cde23-6f1e-47c4-88cb-12df2fc5c590"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:29:52.643547 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.643488 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-workload-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:29:52.643547 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.643499 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1f9cde23-6f1e-47c4-88cb-12df2fc5c590" (UID: "1f9cde23-6f1e-47c4-88cb-12df2fc5c590"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:29:52.644136 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.644099 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1f9cde23-6f1e-47c4-88cb-12df2fc5c590" (UID: "1f9cde23-6f1e-47c4-88cb-12df2fc5c590"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:29:52.644346 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.644298 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-data" (OuterVolumeSpecName: "istio-data") pod "1f9cde23-6f1e-47c4-88cb-12df2fc5c590" (UID: "1f9cde23-6f1e-47c4-88cb-12df2fc5c590"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:29:52.645378 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.645350 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1f9cde23-6f1e-47c4-88cb-12df2fc5c590" (UID: "1f9cde23-6f1e-47c4-88cb-12df2fc5c590"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:29:52.646254 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.646231 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-kube-api-access-tc5r5" (OuterVolumeSpecName: "kube-api-access-tc5r5") pod "1f9cde23-6f1e-47c4-88cb-12df2fc5c590" (UID: "1f9cde23-6f1e-47c4-88cb-12df2fc5c590"). InnerVolumeSpecName "kube-api-access-tc5r5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:29:52.646377 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.646267 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-token" (OuterVolumeSpecName: "istio-token") pod "1f9cde23-6f1e-47c4-88cb-12df2fc5c590" (UID: "1f9cde23-6f1e-47c4-88cb-12df2fc5c590"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:29:52.646564 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.646537 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1f9cde23-6f1e-47c4-88cb-12df2fc5c590" (UID: "1f9cde23-6f1e-47c4-88cb-12df2fc5c590"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:29:52.744849 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.744816 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-workload-certs\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:29:52.744849 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.744846 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istiod-ca-cert\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:29:52.744849 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.744855 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-data\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:29:52.745120 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.744865 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tc5r5\" (UniqueName: \"kubernetes.io/projected/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-kube-api-access-tc5r5\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:29:52.745120 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.744875 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-credential-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:29:52.745120 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.744883 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-envoy\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:29:52.745120 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.744891 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-token\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:29:52.745120 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:52.744898 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f9cde23-6f1e-47c4-88cb-12df2fc5c590-istio-podinfo\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:29:53.102634 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.102591 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5"] Jul 14 01:29:53.103168 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.103145 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9cde23-6f1e-47c4-88cb-12df2fc5c590" containerName="istio-proxy" Jul 14 01:29:53.103260 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.103171 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9cde23-6f1e-47c4-88cb-12df2fc5c590" containerName="istio-proxy" Jul 14 01:29:53.103260 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.103193 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9cde23-6f1e-47c4-88cb-12df2fc5c590" containerName="istio-proxy" Jul 14 01:29:53.103260 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.103203 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9cde23-6f1e-47c4-88cb-12df2fc5c590" containerName="istio-proxy" Jul 14 01:29:53.103462 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.103297 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f9cde23-6f1e-47c4-88cb-12df2fc5c590" containerName="istio-proxy" Jul 14 01:29:53.103462 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.103309 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f9cde23-6f1e-47c4-88cb-12df2fc5c590" containerName="istio-proxy" Jul 14 01:29:53.107894 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.107872 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5" Jul 14 01:29:53.110563 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.110530 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zn7l8\"" Jul 14 01:29:53.110812 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.110796 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 01:29:53.112544 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.112508 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5"] Jul 14 01:29:53.250585 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.250546 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb081f4-12fd-4b80-aa3e-ee861a48dace-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-d3208c92-s25z5\" (UID: \"4eb081f4-12fd-4b80-aa3e-ee861a48dace\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5" Jul 14 01:29:53.351670 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.351633 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb081f4-12fd-4b80-aa3e-ee861a48dace-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-d3208c92-s25z5\" (UID: \"4eb081f4-12fd-4b80-aa3e-ee861a48dace\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5" Jul 14 01:29:53.352345 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.352293 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb081f4-12fd-4b80-aa3e-ee861a48dace-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-d3208c92-s25z5\" (UID: \"4eb081f4-12fd-4b80-aa3e-ee861a48dace\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5" Jul 14 01:29:53.421640 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.421551 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5" Jul 14 01:29:53.520890 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.520854 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" Jul 14 01:29:53.521232 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.520945 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls" event={"ID":"1f9cde23-6f1e-47c4-88cb-12df2fc5c590","Type":"ContainerDied","Data":"69247f3b21e2f77ff29d29112787f12d3111d0a3e56320cd91f2fdab8df235ea"} Jul 14 01:29:53.521232 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.520995 2582 scope.go:117] "RemoveContainer" containerID="9e089b9b3e949c6417f06b702423465eb230a11a51cf7476c74e3f7453288550" Jul 14 01:29:53.553810 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.553779 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls"] Jul 14 01:29:53.555822 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.555797 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e1db3cf4-openshift-default-5494c89b79-nx4ls"] Jul 14 01:29:53.608883 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.608857 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5"] Jul 14 01:29:53.610986 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:29:53.610957 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb081f4_12fd_4b80_aa3e_ee861a48dace.slice/crio-0fbe7033fe829d4c2f0a013fe7b4eae6b108802f2eea2ad18cc896507d5e89f7 WatchSource:0}: Error finding container 0fbe7033fe829d4c2f0a013fe7b4eae6b108802f2eea2ad18cc896507d5e89f7: Status 404 returned error can't find the container with id 0fbe7033fe829d4c2f0a013fe7b4eae6b108802f2eea2ad18cc896507d5e89f7 Jul 14 01:29:53.706690 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:53.706613 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f9cde23-6f1e-47c4-88cb-12df2fc5c590" path="/var/lib/kubelet/pods/1f9cde23-6f1e-47c4-88cb-12df2fc5c590/volumes" Jul 14 01:29:54.528130 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:54.528095 2582 generic.go:356] "Generic (PLEG): container finished" podID="4eb081f4-12fd-4b80-aa3e-ee861a48dace" containerID="456753dd69a955eea46564f321b00ce8f188006fcca0f104a52dc93e08873ae7" exitCode=7 Jul 14 01:29:54.528523 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:54.528146 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5" event={"ID":"4eb081f4-12fd-4b80-aa3e-ee861a48dace","Type":"ContainerDied","Data":"456753dd69a955eea46564f321b00ce8f188006fcca0f104a52dc93e08873ae7"} Jul 14 01:29:54.528523 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:54.528189 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5" event={"ID":"4eb081f4-12fd-4b80-aa3e-ee861a48dace","Type":"ContainerStarted","Data":"0fbe7033fe829d4c2f0a013fe7b4eae6b108802f2eea2ad18cc896507d5e89f7"} Jul 14 01:29:54.528523 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:54.528373 2582 scope.go:117] "RemoveContainer" containerID="456753dd69a955eea46564f321b00ce8f188006fcca0f104a52dc93e08873ae7" Jul 14 01:29:55.536254 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:55.536225 2582 generic.go:356] "Generic (PLEG): container finished" podID="4eb081f4-12fd-4b80-aa3e-ee861a48dace" containerID="e3e8774e64166f282b2924af7e561768debdb212da080eb7e709fa39b0cc052e" exitCode=7 Jul 14 01:29:55.536651 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:55.536303 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5" event={"ID":"4eb081f4-12fd-4b80-aa3e-ee861a48dace","Type":"ContainerDied","Data":"e3e8774e64166f282b2924af7e561768debdb212da080eb7e709fa39b0cc052e"} Jul 14 01:29:55.536651 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:55.536383 2582 scope.go:117] "RemoveContainer" containerID="456753dd69a955eea46564f321b00ce8f188006fcca0f104a52dc93e08873ae7" Jul 14 01:29:55.536651 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:55.536633 2582 scope.go:117] "RemoveContainer" containerID="e3e8774e64166f282b2924af7e561768debdb212da080eb7e709fa39b0cc052e" Jul 14 01:29:55.536884 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:29:55.536857 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-d3208c92-s25z5_odh-ai-gateway-infra(4eb081f4-12fd-4b80-aa3e-ee861a48dace)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5" podUID="4eb081f4-12fd-4b80-aa3e-ee861a48dace" Jul 14 01:29:56.545522 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:56.545492 2582 scope.go:117] "RemoveContainer" containerID="e3e8774e64166f282b2924af7e561768debdb212da080eb7e709fa39b0cc052e" Jul 14 01:29:56.545998 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:29:56.545784 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-d3208c92-s25z5_odh-ai-gateway-infra(4eb081f4-12fd-4b80-aa3e-ee861a48dace)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5" podUID="4eb081f4-12fd-4b80-aa3e-ee861a48dace" Jul 14 01:29:59.562739 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:59.562651 2582 generic.go:356] "Generic (PLEG): container finished" podID="9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" containerID="6aaf60f3731c9cfb98128bc9cf0de6a694fbe10408a4364d8c873a2944f2f980" exitCode=0 Jul 14 01:29:59.563123 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:59.562736 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" event={"ID":"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e","Type":"ContainerDied","Data":"6aaf60f3731c9cfb98128bc9cf0de6a694fbe10408a4364d8c873a2944f2f980"} Jul 14 01:29:59.563177 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:59.563133 2582 scope.go:117] "RemoveContainer" containerID="6aaf60f3731c9cfb98128bc9cf0de6a694fbe10408a4364d8c873a2944f2f980" Jul 14 01:29:59.564169 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:59.564142 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:29:59.564230 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:59.564210 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:29:59.564262 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:29:59.564241 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:30:00.133914 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:00.133873 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733210-4cs6x"] Jul 14 01:30:00.137834 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:00.137815 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-4cs6x" Jul 14 01:30:00.144785 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:00.144756 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733210-4cs6x"] Jul 14 01:30:00.151557 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:00.151532 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-4cs6x" Jul 14 01:30:00.338157 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:00.338131 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733210-4cs6x"] Jul 14 01:30:00.340525 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:30:00.340489 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3513846e_02de_4243_83fc_ca6ae1414edd.slice/crio-4122ef397ef33e030395951bff3b32cec845b51896a9cabb9db81987d88c1ac6 WatchSource:0}: Error finding container 4122ef397ef33e030395951bff3b32cec845b51896a9cabb9db81987d88c1ac6: Status 404 returned error can't find the container with id 4122ef397ef33e030395951bff3b32cec845b51896a9cabb9db81987d88c1ac6 Jul 14 01:30:00.569672 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:00.569559 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-4cs6x" event={"ID":"3513846e-02de-4243-83fc-ca6ae1414edd","Type":"ContainerStarted","Data":"4122ef397ef33e030395951bff3b32cec845b51896a9cabb9db81987d88c1ac6"} Jul 14 01:30:00.571578 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:00.571552 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" event={"ID":"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e","Type":"ContainerStarted","Data":"10e12e96a009edef098c07ba7540d3223ed6cea8d17b8cfd9cc99bc78acf7d9e"} Jul 14 01:30:00.870522 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:00.870484 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:30:00.871549 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:00.871527 2582 patch_prober.go:28] interesting pod/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 14 01:30:00.871607 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:00.871576 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" podUID="9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 14 01:30:01.581789 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:01.581753 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-4cs6x" event={"ID":"3513846e-02de-4243-83fc-ca6ae1414edd","Type":"ContainerStarted","Data":"7eb6646bafd4a530bd83d308f040ba0e67dd552ad2a54a011d1d6cc686017b72"} Jul 14 01:30:01.598479 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:01.598417 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-4cs6x" podStartSLOduration=1.598380228 podStartE2EDuration="1.598380228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:30:01.596407762 +0000 UTC m=+1830.499263262" watchObservedRunningTime="2026-07-14 01:30:01.598380228 +0000 UTC m=+1830.501235727" Jul 14 01:30:01.871383 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:01.871308 2582 patch_prober.go:28] interesting pod/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 14 01:30:01.871559 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:01.871411 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" podUID="9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 14 01:30:02.590922 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:02.590880 2582 generic.go:356] "Generic (PLEG): container finished" podID="3513846e-02de-4243-83fc-ca6ae1414edd" containerID="7eb6646bafd4a530bd83d308f040ba0e67dd552ad2a54a011d1d6cc686017b72" exitCode=0 Jul 14 01:30:02.591440 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:02.590991 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-4cs6x" event={"ID":"3513846e-02de-4243-83fc-ca6ae1414edd","Type":"ContainerDied","Data":"7eb6646bafd4a530bd83d308f040ba0e67dd552ad2a54a011d1d6cc686017b72"} Jul 14 01:30:02.871674 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:02.871583 2582 patch_prober.go:28] interesting pod/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 14 01:30:02.871674 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:02.871638 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" podUID="9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 14 01:30:03.760686 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:03.760663 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-4cs6x" Jul 14 01:30:03.870755 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:03.870672 2582 patch_prober.go:28] interesting pod/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 14 01:30:03.870755 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:03.870729 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" podUID="9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 14 01:30:04.606670 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:04.606637 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-4cs6x" Jul 14 01:30:04.606670 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:04.606649 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733210-4cs6x" event={"ID":"3513846e-02de-4243-83fc-ca6ae1414edd","Type":"ContainerDied","Data":"4122ef397ef33e030395951bff3b32cec845b51896a9cabb9db81987d88c1ac6"} Jul 14 01:30:04.606895 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:04.606718 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4122ef397ef33e030395951bff3b32cec845b51896a9cabb9db81987d88c1ac6" Jul 14 01:30:04.874959 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:04.874853 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:30:04.875633 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:04.875171 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:30:04.875992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:04.875974 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:30:07.687371 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:07.687339 2582 scope.go:117] "RemoveContainer" containerID="e3e8774e64166f282b2924af7e561768debdb212da080eb7e709fa39b0cc052e" Jul 14 01:30:08.630165 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:08.630072 2582 generic.go:356] "Generic (PLEG): container finished" podID="4eb081f4-12fd-4b80-aa3e-ee861a48dace" containerID="9723f327df3f88b369b5a651efb0d2e92409546438a1bdc095a66c911760d72a" exitCode=0 Jul 14 01:30:08.630165 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:08.630146 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5" event={"ID":"4eb081f4-12fd-4b80-aa3e-ee861a48dace","Type":"ContainerDied","Data":"9723f327df3f88b369b5a651efb0d2e92409546438a1bdc095a66c911760d72a"} Jul 14 01:30:08.630427 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:08.630219 2582 scope.go:117] "RemoveContainer" containerID="e3e8774e64166f282b2924af7e561768debdb212da080eb7e709fa39b0cc052e" Jul 14 01:30:09.662431 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:09.662393 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5"] Jul 14 01:30:09.768025 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:09.768001 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5" Jul 14 01:30:09.812351 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:09.812300 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb081f4-12fd-4b80-aa3e-ee861a48dace-maas-api-service-ca\") pod \"4eb081f4-12fd-4b80-aa3e-ee861a48dace\" (UID: \"4eb081f4-12fd-4b80-aa3e-ee861a48dace\") " Jul 14 01:30:09.812670 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:09.812649 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eb081f4-12fd-4b80-aa3e-ee861a48dace-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4eb081f4-12fd-4b80-aa3e-ee861a48dace" (UID: "4eb081f4-12fd-4b80-aa3e-ee861a48dace"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:30:09.913134 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:09.913049 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb081f4-12fd-4b80-aa3e-ee861a48dace-maas-api-service-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:30:10.643449 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:10.643407 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5" Jul 14 01:30:10.643643 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:10.643411 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5" event={"ID":"4eb081f4-12fd-4b80-aa3e-ee861a48dace","Type":"ContainerDied","Data":"0fbe7033fe829d4c2f0a013fe7b4eae6b108802f2eea2ad18cc896507d5e89f7"} Jul 14 01:30:10.643643 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:10.643551 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0fbe7033fe829d4c2f0a013fe7b4eae6b108802f2eea2ad18cc896507d5e89f7" Jul 14 01:30:10.674665 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:10.674632 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5"] Jul 14 01:30:10.678517 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:10.678490 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d3208c92-s25z5"] Jul 14 01:30:11.694466 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:30:11.694432 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eb081f4-12fd-4b80-aa3e-ee861a48dace" path="/var/lib/kubelet/pods/4eb081f4-12fd-4b80-aa3e-ee861a48dace/volumes" Jul 14 01:31:08.376527 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:08.376484 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5"] Jul 14 01:31:08.377040 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:08.376766 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" podUID="9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" containerName="istio-proxy" containerID="cri-o://10e12e96a009edef098c07ba7540d3223ed6cea8d17b8cfd9cc99bc78acf7d9e" gracePeriod=30 Jul 14 01:31:13.634345 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.634303 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:31:13.717212 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.717183 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-token\") pod \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " Jul 14 01:31:13.717377 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.717232 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-credential-socket\") pod \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " Jul 14 01:31:13.717377 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.717277 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istiod-ca-cert\") pod \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " Jul 14 01:31:13.717377 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.717315 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-workload-certs\") pod \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " Jul 14 01:31:13.717377 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.717366 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-data\") pod \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " Jul 14 01:31:13.717590 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.717415 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bd22z\" (UniqueName: \"kubernetes.io/projected/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-kube-api-access-bd22z\") pod \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " Jul 14 01:31:13.717590 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.717461 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-envoy\") pod \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " Jul 14 01:31:13.717590 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.717497 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-podinfo\") pod \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " Jul 14 01:31:13.717590 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.717529 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-workload-socket\") pod \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\" (UID: \"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e\") " Jul 14 01:31:13.717590 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.717571 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" (UID: "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:31:13.717848 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.717797 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-credential-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:31:13.717848 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.717817 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" (UID: "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:31:13.718170 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.717768 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" (UID: "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:31:13.718170 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.718001 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" (UID: "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:31:13.718621 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.718589 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-data" (OuterVolumeSpecName: "istio-data") pod "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" (UID: "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:31:13.719712 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.719672 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-token" (OuterVolumeSpecName: "istio-token") pod "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" (UID: "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:31:13.720050 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.720017 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" (UID: "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:31:13.720150 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.720055 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-kube-api-access-bd22z" (OuterVolumeSpecName: "kube-api-access-bd22z") pod "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" (UID: "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e"). InnerVolumeSpecName "kube-api-access-bd22z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:31:13.720375 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.720347 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" (UID: "9afbdbda-990a-4c4c-adfc-4018f5fd1a7e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:31:13.818814 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.818782 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-envoy\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:31:13.818814 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.818807 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-podinfo\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:31:13.818814 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.818817 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-workload-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:31:13.819043 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.818826 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-token\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:31:13.819043 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.818835 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istiod-ca-cert\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:31:13.819043 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.818842 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-workload-certs\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:31:13.819043 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.818849 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-istio-data\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:31:13.819043 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.818857 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd22z\" (UniqueName: \"kubernetes.io/projected/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e-kube-api-access-bd22z\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:31:13.971949 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.971850 2582 generic.go:356] "Generic (PLEG): container finished" podID="9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" containerID="10e12e96a009edef098c07ba7540d3223ed6cea8d17b8cfd9cc99bc78acf7d9e" exitCode=0 Jul 14 01:31:13.971949 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.971909 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" Jul 14 01:31:13.972232 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.971938 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" event={"ID":"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e","Type":"ContainerDied","Data":"10e12e96a009edef098c07ba7540d3223ed6cea8d17b8cfd9cc99bc78acf7d9e"} Jul 14 01:31:13.972232 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.971998 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5" event={"ID":"9afbdbda-990a-4c4c-adfc-4018f5fd1a7e","Type":"ContainerDied","Data":"0022e256e861bc14157d44854ba2e56cca6906599202fd4f20f07affbb103752"} Jul 14 01:31:13.972232 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.972024 2582 scope.go:117] "RemoveContainer" containerID="10e12e96a009edef098c07ba7540d3223ed6cea8d17b8cfd9cc99bc78acf7d9e" Jul 14 01:31:13.986379 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.986353 2582 scope.go:117] "RemoveContainer" containerID="6aaf60f3731c9cfb98128bc9cf0de6a694fbe10408a4364d8c873a2944f2f980" Jul 14 01:31:13.994747 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.994726 2582 scope.go:117] "RemoveContainer" containerID="10e12e96a009edef098c07ba7540d3223ed6cea8d17b8cfd9cc99bc78acf7d9e" Jul 14 01:31:13.995052 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:31:13.995032 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10e12e96a009edef098c07ba7540d3223ed6cea8d17b8cfd9cc99bc78acf7d9e\": container with ID starting with 10e12e96a009edef098c07ba7540d3223ed6cea8d17b8cfd9cc99bc78acf7d9e not found: ID does not exist" containerID="10e12e96a009edef098c07ba7540d3223ed6cea8d17b8cfd9cc99bc78acf7d9e" Jul 14 01:31:13.995125 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.995061 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10e12e96a009edef098c07ba7540d3223ed6cea8d17b8cfd9cc99bc78acf7d9e"} err="failed to get container status \"10e12e96a009edef098c07ba7540d3223ed6cea8d17b8cfd9cc99bc78acf7d9e\": rpc error: code = NotFound desc = could not find container \"10e12e96a009edef098c07ba7540d3223ed6cea8d17b8cfd9cc99bc78acf7d9e\": container with ID starting with 10e12e96a009edef098c07ba7540d3223ed6cea8d17b8cfd9cc99bc78acf7d9e not found: ID does not exist" Jul 14 01:31:13.995125 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.995079 2582 scope.go:117] "RemoveContainer" containerID="6aaf60f3731c9cfb98128bc9cf0de6a694fbe10408a4364d8c873a2944f2f980" Jul 14 01:31:13.995535 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:31:13.995515 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6aaf60f3731c9cfb98128bc9cf0de6a694fbe10408a4364d8c873a2944f2f980\": container with ID starting with 6aaf60f3731c9cfb98128bc9cf0de6a694fbe10408a4364d8c873a2944f2f980 not found: ID does not exist" containerID="6aaf60f3731c9cfb98128bc9cf0de6a694fbe10408a4364d8c873a2944f2f980" Jul 14 01:31:13.995535 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.995539 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6aaf60f3731c9cfb98128bc9cf0de6a694fbe10408a4364d8c873a2944f2f980"} err="failed to get container status \"6aaf60f3731c9cfb98128bc9cf0de6a694fbe10408a4364d8c873a2944f2f980\": rpc error: code = NotFound desc = could not find container \"6aaf60f3731c9cfb98128bc9cf0de6a694fbe10408a4364d8c873a2944f2f980\": container with ID starting with 6aaf60f3731c9cfb98128bc9cf0de6a694fbe10408a4364d8c873a2944f2f980 not found: ID does not exist" Jul 14 01:31:13.998906 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:13.998845 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5"] Jul 14 01:31:14.000738 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:14.000719 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-d3208c92-openshift-default-c695f8d79-qbfr5"] Jul 14 01:31:15.693150 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:31:15.693116 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" path="/var/lib/kubelet/pods/9afbdbda-990a-4c4c-adfc-4018f5fd1a7e/volumes" Jul 14 01:32:43.254828 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.254728 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc"] Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255092 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" containerName="istio-proxy" Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255104 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" containerName="istio-proxy" Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255113 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eb081f4-12fd-4b80-aa3e-ee861a48dace" containerName="revoke-keys" Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255118 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eb081f4-12fd-4b80-aa3e-ee861a48dace" containerName="revoke-keys" Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255124 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eb081f4-12fd-4b80-aa3e-ee861a48dace" containerName="revoke-keys" Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255129 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eb081f4-12fd-4b80-aa3e-ee861a48dace" containerName="revoke-keys" Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255144 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3513846e-02de-4243-83fc-ca6ae1414edd" containerName="cleanup" Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255150 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3513846e-02de-4243-83fc-ca6ae1414edd" containerName="cleanup" Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255160 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" containerName="istio-proxy" Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255165 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" containerName="istio-proxy" Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255172 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eb081f4-12fd-4b80-aa3e-ee861a48dace" containerName="revoke-keys" Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255177 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eb081f4-12fd-4b80-aa3e-ee861a48dace" containerName="revoke-keys" Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255228 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eb081f4-12fd-4b80-aa3e-ee861a48dace" containerName="revoke-keys" Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255236 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3513846e-02de-4243-83fc-ca6ae1414edd" containerName="cleanup" Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255242 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" containerName="istio-proxy" Jul 14 01:32:43.255363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.255249 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eb081f4-12fd-4b80-aa3e-ee861a48dace" containerName="revoke-keys" Jul 14 01:32:43.258250 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.258232 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.260723 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.260703 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-40103c45-openshift-default-dockercfg-vj9wj\"" Jul 14 01:32:43.275176 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.275149 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc"] Jul 14 01:32:43.429199 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.429166 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-istio-envoy\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.429199 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.429204 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1da63730-2466-4411-b76c-b69221b7ec5a-istio-podinfo\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.429503 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.429235 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-workload-certs\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.429503 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.429345 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1da63730-2466-4411-b76c-b69221b7ec5a-istiod-ca-cert\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.429503 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.429398 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-credential-socket\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.429503 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.429478 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1da63730-2466-4411-b76c-b69221b7ec5a-istio-token\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.429677 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.429504 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-istio-data\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.429677 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.429527 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5ldz\" (UniqueName: \"kubernetes.io/projected/1da63730-2466-4411-b76c-b69221b7ec5a-kube-api-access-g5ldz\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.429677 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.429587 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-workload-socket\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.530960 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.530856 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-workload-certs\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.530960 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.530924 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1da63730-2466-4411-b76c-b69221b7ec5a-istiod-ca-cert\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.530960 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.530956 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-credential-socket\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.531259 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.531057 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1da63730-2466-4411-b76c-b69221b7ec5a-istio-token\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.531259 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.531084 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-istio-data\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.531259 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.531109 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5ldz\" (UniqueName: \"kubernetes.io/projected/1da63730-2466-4411-b76c-b69221b7ec5a-kube-api-access-g5ldz\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.531259 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.531146 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-workload-socket\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.531259 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.531183 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-istio-envoy\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.531259 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.531215 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1da63730-2466-4411-b76c-b69221b7ec5a-istio-podinfo\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.531624 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.531377 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-workload-certs\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.531624 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.531481 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-istio-data\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.531624 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.531500 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-credential-socket\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.531790 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.531718 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-workload-socket\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.532045 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.532023 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1da63730-2466-4411-b76c-b69221b7ec5a-istiod-ca-cert\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.533707 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.533678 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-istio-envoy\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.534101 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.534085 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1da63730-2466-4411-b76c-b69221b7ec5a-istio-podinfo\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.539567 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.539540 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1da63730-2466-4411-b76c-b69221b7ec5a-istio-token\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.539697 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.539671 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5ldz\" (UniqueName: \"kubernetes.io/projected/1da63730-2466-4411-b76c-b69221b7ec5a-kube-api-access-g5ldz\") pod \"e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.569538 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.569501 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:43.723541 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.723466 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc"] Jul 14 01:32:43.726026 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:32:43.725997 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1da63730_2466_4411_b76c_b69221b7ec5a.slice/crio-5d535d8cf7c745bb0d68cc5b01e6072a3a349643457195a138a6b3ea19504cb0 WatchSource:0}: Error finding container 5d535d8cf7c745bb0d68cc5b01e6072a3a349643457195a138a6b3ea19504cb0: Status 404 returned error can't find the container with id 5d535d8cf7c745bb0d68cc5b01e6072a3a349643457195a138a6b3ea19504cb0 Jul 14 01:32:43.728202 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.728167 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:32:43.728305 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.728254 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:32:43.728305 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:43.728296 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:32:44.426191 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:44.426149 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" event={"ID":"1da63730-2466-4411-b76c-b69221b7ec5a","Type":"ContainerStarted","Data":"b719fb8602791c1536c296071bbda8b12a76d98b160c4bd1771df959d15cc4ee"} Jul 14 01:32:44.426582 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:44.426215 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" event={"ID":"1da63730-2466-4411-b76c-b69221b7ec5a","Type":"ContainerStarted","Data":"5d535d8cf7c745bb0d68cc5b01e6072a3a349643457195a138a6b3ea19504cb0"} Jul 14 01:32:44.446866 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:44.446807 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" podStartSLOduration=1.44678996 podStartE2EDuration="1.44678996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:32:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:32:44.44441459 +0000 UTC m=+1993.347270090" watchObservedRunningTime="2026-07-14 01:32:44.44678996 +0000 UTC m=+1993.349645458" Jul 14 01:32:44.570591 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:44.570558 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:44.575401 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:44.575375 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:45.431378 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:45.431346 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:45.432358 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:45.432317 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:32:58.129094 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:58.129045 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40103c45-fr629"] Jul 14 01:32:58.129747 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:58.129725 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9afbdbda-990a-4c4c-adfc-4018f5fd1a7e" containerName="istio-proxy" Jul 14 01:32:58.129791 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:58.129759 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eb081f4-12fd-4b80-aa3e-ee861a48dace" containerName="revoke-keys" Jul 14 01:32:58.133496 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:58.133477 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40103c45-fr629" Jul 14 01:32:58.136220 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:58.135991 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zn7l8\"" Jul 14 01:32:58.136220 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:58.136029 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 01:32:58.138362 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:58.138305 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40103c45-fr629"] Jul 14 01:32:58.147778 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:58.147739 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/325bef2d-f63f-4786-80f8-66e1227b000d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-40103c45-fr629\" (UID: \"325bef2d-f63f-4786-80f8-66e1227b000d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40103c45-fr629" Jul 14 01:32:58.249504 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:58.249446 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/325bef2d-f63f-4786-80f8-66e1227b000d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-40103c45-fr629\" (UID: \"325bef2d-f63f-4786-80f8-66e1227b000d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40103c45-fr629" Jul 14 01:32:58.250369 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:58.250315 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/325bef2d-f63f-4786-80f8-66e1227b000d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-40103c45-fr629\" (UID: \"325bef2d-f63f-4786-80f8-66e1227b000d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40103c45-fr629" Jul 14 01:32:58.448143 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:58.448056 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40103c45-fr629" Jul 14 01:32:58.593154 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:58.593121 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40103c45-fr629"] Jul 14 01:32:58.595083 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:32:58.595056 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod325bef2d_f63f_4786_80f8_66e1227b000d.slice/crio-e27520d1ca228c63fc8091567aaffe843106ba106fab4996548de2984424cdc6 WatchSource:0}: Error finding container e27520d1ca228c63fc8091567aaffe843106ba106fab4996548de2984424cdc6: Status 404 returned error can't find the container with id e27520d1ca228c63fc8091567aaffe843106ba106fab4996548de2984424cdc6 Jul 14 01:32:59.516116 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:59.516081 2582 generic.go:356] "Generic (PLEG): container finished" podID="325bef2d-f63f-4786-80f8-66e1227b000d" containerID="bed830e301728414750b913d94a100b230981c8236dd0ea48a61eb2f9fc5ac62" exitCode=0 Jul 14 01:32:59.516535 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:59.516172 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40103c45-fr629" event={"ID":"325bef2d-f63f-4786-80f8-66e1227b000d","Type":"ContainerDied","Data":"bed830e301728414750b913d94a100b230981c8236dd0ea48a61eb2f9fc5ac62"} Jul 14 01:32:59.516535 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:32:59.516226 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40103c45-fr629" event={"ID":"325bef2d-f63f-4786-80f8-66e1227b000d","Type":"ContainerStarted","Data":"e27520d1ca228c63fc8091567aaffe843106ba106fab4996548de2984424cdc6"} Jul 14 01:33:00.433356 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:00.433308 2582 patch_prober.go:28] interesting pod/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 01:33:00.433563 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:00.433376 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" podUID="1da63730-2466-4411-b76c-b69221b7ec5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 01:33:00.522862 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:00.522829 2582 generic.go:356] "Generic (PLEG): container finished" podID="1da63730-2466-4411-b76c-b69221b7ec5a" containerID="b719fb8602791c1536c296071bbda8b12a76d98b160c4bd1771df959d15cc4ee" exitCode=0 Jul 14 01:33:00.523293 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:00.522911 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" event={"ID":"1da63730-2466-4411-b76c-b69221b7ec5a","Type":"ContainerDied","Data":"b719fb8602791c1536c296071bbda8b12a76d98b160c4bd1771df959d15cc4ee"} Jul 14 01:33:00.523665 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:00.523461 2582 scope.go:117] "RemoveContainer" containerID="b719fb8602791c1536c296071bbda8b12a76d98b160c4bd1771df959d15cc4ee" Jul 14 01:33:00.524501 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:00.524462 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:33:00.524581 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:00.524555 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:33:00.524618 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:00.524590 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:33:00.681456 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:00.681428 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40103c45-fr629" Jul 14 01:33:00.774385 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:00.774265 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/325bef2d-f63f-4786-80f8-66e1227b000d-maas-api-service-ca\") pod \"325bef2d-f63f-4786-80f8-66e1227b000d\" (UID: \"325bef2d-f63f-4786-80f8-66e1227b000d\") " Jul 14 01:33:00.774894 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:00.774864 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/325bef2d-f63f-4786-80f8-66e1227b000d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "325bef2d-f63f-4786-80f8-66e1227b000d" (UID: "325bef2d-f63f-4786-80f8-66e1227b000d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:33:00.875252 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:00.875218 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/325bef2d-f63f-4786-80f8-66e1227b000d-maas-api-service-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:33:01.535806 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:01.535764 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" event={"ID":"1da63730-2466-4411-b76c-b69221b7ec5a","Type":"ContainerStarted","Data":"627c7a878a22dc4e0ac1ee07f2ae5f657b87638a2fac500f108535460e1c343d"} Jul 14 01:33:01.537198 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:01.537171 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40103c45-fr629" event={"ID":"325bef2d-f63f-4786-80f8-66e1227b000d","Type":"ContainerDied","Data":"e27520d1ca228c63fc8091567aaffe843106ba106fab4996548de2984424cdc6"} Jul 14 01:33:01.537310 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:01.537216 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e27520d1ca228c63fc8091567aaffe843106ba106fab4996548de2984424cdc6" Jul 14 01:33:01.537310 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:01.537186 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-40103c45-fr629" Jul 14 01:33:01.569890 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:01.569857 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:33:01.570092 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:01.570072 2582 patch_prober.go:28] interesting pod/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 01:33:01.570137 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:01.570118 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" podUID="1da63730-2466-4411-b76c-b69221b7ec5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 01:33:02.570397 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:02.570318 2582 patch_prober.go:28] interesting pod/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 01:33:02.570799 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:02.570416 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" podUID="1da63730-2466-4411-b76c-b69221b7ec5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 01:33:03.569956 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:03.569918 2582 patch_prober.go:28] interesting pod/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 01:33:03.570143 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:03.569974 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" podUID="1da63730-2466-4411-b76c-b69221b7ec5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 01:33:04.569990 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:04.569951 2582 patch_prober.go:28] interesting pod/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 01:33:04.570514 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:04.570022 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" podUID="1da63730-2466-4411-b76c-b69221b7ec5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 01:33:05.573812 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:05.573781 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:33:05.574252 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:05.574045 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:33:05.574748 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:05.574734 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:33:23.853916 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:23.853881 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc"] Jul 14 01:33:23.854547 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:23.854133 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" podUID="1da63730-2466-4411-b76c-b69221b7ec5a" containerName="istio-proxy" containerID="cri-o://627c7a878a22dc4e0ac1ee07f2ae5f657b87638a2fac500f108535460e1c343d" gracePeriod=30 Jul 14 01:33:29.109302 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.109275 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:33:29.243624 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.243533 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-workload-certs\") pod \"1da63730-2466-4411-b76c-b69221b7ec5a\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " Jul 14 01:33:29.243624 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.243575 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1da63730-2466-4411-b76c-b69221b7ec5a-istiod-ca-cert\") pod \"1da63730-2466-4411-b76c-b69221b7ec5a\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " Jul 14 01:33:29.243624 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.243606 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5ldz\" (UniqueName: \"kubernetes.io/projected/1da63730-2466-4411-b76c-b69221b7ec5a-kube-api-access-g5ldz\") pod \"1da63730-2466-4411-b76c-b69221b7ec5a\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " Jul 14 01:33:29.243885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.243657 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-credential-socket\") pod \"1da63730-2466-4411-b76c-b69221b7ec5a\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " Jul 14 01:33:29.243885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.243682 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-workload-socket\") pod \"1da63730-2466-4411-b76c-b69221b7ec5a\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " Jul 14 01:33:29.243885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.243744 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-istio-envoy\") pod \"1da63730-2466-4411-b76c-b69221b7ec5a\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " Jul 14 01:33:29.243885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.243768 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1da63730-2466-4411-b76c-b69221b7ec5a-istio-token\") pod \"1da63730-2466-4411-b76c-b69221b7ec5a\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " Jul 14 01:33:29.243885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.243793 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-istio-data\") pod \"1da63730-2466-4411-b76c-b69221b7ec5a\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " Jul 14 01:33:29.243885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.243825 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1da63730-2466-4411-b76c-b69221b7ec5a-istio-podinfo\") pod \"1da63730-2466-4411-b76c-b69221b7ec5a\" (UID: \"1da63730-2466-4411-b76c-b69221b7ec5a\") " Jul 14 01:33:29.243885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.243875 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1da63730-2466-4411-b76c-b69221b7ec5a" (UID: "1da63730-2466-4411-b76c-b69221b7ec5a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:33:29.244248 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.244036 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1da63730-2466-4411-b76c-b69221b7ec5a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1da63730-2466-4411-b76c-b69221b7ec5a" (UID: "1da63730-2466-4411-b76c-b69221b7ec5a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:33:29.244248 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.244099 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-workload-certs\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:33:29.244248 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.244037 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1da63730-2466-4411-b76c-b69221b7ec5a" (UID: "1da63730-2466-4411-b76c-b69221b7ec5a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:33:29.244433 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.244353 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1da63730-2466-4411-b76c-b69221b7ec5a" (UID: "1da63730-2466-4411-b76c-b69221b7ec5a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:33:29.244743 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.244724 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-istio-data" (OuterVolumeSpecName: "istio-data") pod "1da63730-2466-4411-b76c-b69221b7ec5a" (UID: "1da63730-2466-4411-b76c-b69221b7ec5a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:33:29.246752 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.246727 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1da63730-2466-4411-b76c-b69221b7ec5a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1da63730-2466-4411-b76c-b69221b7ec5a" (UID: "1da63730-2466-4411-b76c-b69221b7ec5a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:33:29.246752 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.246733 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1da63730-2466-4411-b76c-b69221b7ec5a-istio-token" (OuterVolumeSpecName: "istio-token") pod "1da63730-2466-4411-b76c-b69221b7ec5a" (UID: "1da63730-2466-4411-b76c-b69221b7ec5a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:33:29.246752 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.246745 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1da63730-2466-4411-b76c-b69221b7ec5a-kube-api-access-g5ldz" (OuterVolumeSpecName: "kube-api-access-g5ldz") pod "1da63730-2466-4411-b76c-b69221b7ec5a" (UID: "1da63730-2466-4411-b76c-b69221b7ec5a"). InnerVolumeSpecName "kube-api-access-g5ldz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:33:29.246995 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.246803 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1da63730-2466-4411-b76c-b69221b7ec5a" (UID: "1da63730-2466-4411-b76c-b69221b7ec5a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:33:29.344865 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.344827 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1da63730-2466-4411-b76c-b69221b7ec5a-istiod-ca-cert\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:33:29.344865 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.344861 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5ldz\" (UniqueName: \"kubernetes.io/projected/1da63730-2466-4411-b76c-b69221b7ec5a-kube-api-access-g5ldz\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:33:29.344865 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.344871 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-credential-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:33:29.345124 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.344880 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-workload-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:33:29.345124 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.344890 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-istio-envoy\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:33:29.345124 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.344898 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1da63730-2466-4411-b76c-b69221b7ec5a-istio-token\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:33:29.345124 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.344906 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1da63730-2466-4411-b76c-b69221b7ec5a-istio-data\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:33:29.345124 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.344913 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1da63730-2466-4411-b76c-b69221b7ec5a-istio-podinfo\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:33:29.680424 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.680381 2582 generic.go:356] "Generic (PLEG): container finished" podID="1da63730-2466-4411-b76c-b69221b7ec5a" containerID="627c7a878a22dc4e0ac1ee07f2ae5f657b87638a2fac500f108535460e1c343d" exitCode=0 Jul 14 01:33:29.680630 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.680441 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" Jul 14 01:33:29.680630 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.680437 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" event={"ID":"1da63730-2466-4411-b76c-b69221b7ec5a","Type":"ContainerDied","Data":"627c7a878a22dc4e0ac1ee07f2ae5f657b87638a2fac500f108535460e1c343d"} Jul 14 01:33:29.680630 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.680582 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc" event={"ID":"1da63730-2466-4411-b76c-b69221b7ec5a","Type":"ContainerDied","Data":"5d535d8cf7c745bb0d68cc5b01e6072a3a349643457195a138a6b3ea19504cb0"} Jul 14 01:33:29.680630 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.680619 2582 scope.go:117] "RemoveContainer" containerID="627c7a878a22dc4e0ac1ee07f2ae5f657b87638a2fac500f108535460e1c343d" Jul 14 01:33:29.691044 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.691023 2582 scope.go:117] "RemoveContainer" containerID="b719fb8602791c1536c296071bbda8b12a76d98b160c4bd1771df959d15cc4ee" Jul 14 01:33:29.701175 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.701150 2582 scope.go:117] "RemoveContainer" containerID="627c7a878a22dc4e0ac1ee07f2ae5f657b87638a2fac500f108535460e1c343d" Jul 14 01:33:29.701476 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:33:29.701454 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"627c7a878a22dc4e0ac1ee07f2ae5f657b87638a2fac500f108535460e1c343d\": container with ID starting with 627c7a878a22dc4e0ac1ee07f2ae5f657b87638a2fac500f108535460e1c343d not found: ID does not exist" containerID="627c7a878a22dc4e0ac1ee07f2ae5f657b87638a2fac500f108535460e1c343d" Jul 14 01:33:29.701564 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.701486 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"627c7a878a22dc4e0ac1ee07f2ae5f657b87638a2fac500f108535460e1c343d"} err="failed to get container status \"627c7a878a22dc4e0ac1ee07f2ae5f657b87638a2fac500f108535460e1c343d\": rpc error: code = NotFound desc = could not find container \"627c7a878a22dc4e0ac1ee07f2ae5f657b87638a2fac500f108535460e1c343d\": container with ID starting with 627c7a878a22dc4e0ac1ee07f2ae5f657b87638a2fac500f108535460e1c343d not found: ID does not exist" Jul 14 01:33:29.701564 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.701507 2582 scope.go:117] "RemoveContainer" containerID="b719fb8602791c1536c296071bbda8b12a76d98b160c4bd1771df959d15cc4ee" Jul 14 01:33:29.701789 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:33:29.701770 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b719fb8602791c1536c296071bbda8b12a76d98b160c4bd1771df959d15cc4ee\": container with ID starting with b719fb8602791c1536c296071bbda8b12a76d98b160c4bd1771df959d15cc4ee not found: ID does not exist" containerID="b719fb8602791c1536c296071bbda8b12a76d98b160c4bd1771df959d15cc4ee" Jul 14 01:33:29.701836 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.701797 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b719fb8602791c1536c296071bbda8b12a76d98b160c4bd1771df959d15cc4ee"} err="failed to get container status \"b719fb8602791c1536c296071bbda8b12a76d98b160c4bd1771df959d15cc4ee\": rpc error: code = NotFound desc = could not find container \"b719fb8602791c1536c296071bbda8b12a76d98b160c4bd1771df959d15cc4ee\": container with ID starting with b719fb8602791c1536c296071bbda8b12a76d98b160c4bd1771df959d15cc4ee not found: ID does not exist" Jul 14 01:33:29.702234 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.702216 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc"] Jul 14 01:33:29.707310 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:29.707285 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-40103c45-openshift-default-58bf69c65f-k5xgc"] Jul 14 01:33:31.693804 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:31.693768 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1da63730-2466-4411-b76c-b69221b7ec5a" path="/var/lib/kubelet/pods/1da63730-2466-4411-b76c-b69221b7ec5a/volumes" Jul 14 01:33:31.788537 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:31.788506 2582 scope.go:117] "RemoveContainer" containerID="b49bab9e6e7cd74411cf26118f17b1bdacc970f348997f36056e54129ea315b8" Jul 14 01:33:43.146486 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.146447 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp"] Jul 14 01:33:43.146988 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.146967 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="325bef2d-f63f-4786-80f8-66e1227b000d" containerName="revoke-keys" Jul 14 01:33:43.147065 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.146992 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="325bef2d-f63f-4786-80f8-66e1227b000d" containerName="revoke-keys" Jul 14 01:33:43.147065 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.147010 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da63730-2466-4411-b76c-b69221b7ec5a" containerName="istio-proxy" Jul 14 01:33:43.147065 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.147018 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da63730-2466-4411-b76c-b69221b7ec5a" containerName="istio-proxy" Jul 14 01:33:43.147065 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.147029 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da63730-2466-4411-b76c-b69221b7ec5a" containerName="istio-proxy" Jul 14 01:33:43.147065 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.147037 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da63730-2466-4411-b76c-b69221b7ec5a" containerName="istio-proxy" Jul 14 01:33:43.147305 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.147170 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da63730-2466-4411-b76c-b69221b7ec5a" containerName="istio-proxy" Jul 14 01:33:43.147305 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.147184 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="325bef2d-f63f-4786-80f8-66e1227b000d" containerName="revoke-keys" Jul 14 01:33:43.150632 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.150602 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.153403 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.153382 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-45ee56e2-openshift-default-dockercfg-7smqw\"" Jul 14 01:33:43.160546 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.160521 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp"] Jul 14 01:33:43.165603 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.165578 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-data\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.165749 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.165634 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmb6d\" (UniqueName: \"kubernetes.io/projected/e48eac13-a37f-4508-a5c5-b5f27b86abc8-kube-api-access-kmb6d\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.165749 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.165666 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istiod-ca-cert\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.165749 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.165707 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-workload-socket\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.165749 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.165722 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-workload-certs\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.165970 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.165763 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-envoy\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.165970 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.165817 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-podinfo\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.165970 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.165849 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-credential-socket\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.165970 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.165878 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-token\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.267130 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.267091 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmb6d\" (UniqueName: \"kubernetes.io/projected/e48eac13-a37f-4508-a5c5-b5f27b86abc8-kube-api-access-kmb6d\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.267289 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.267145 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istiod-ca-cert\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.267289 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.267205 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-workload-socket\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.267289 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.267232 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-workload-certs\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.267289 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.267254 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-envoy\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.267289 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.267277 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-podinfo\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.267584 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.267351 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-credential-socket\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.267584 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.267383 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-token\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.267584 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.267418 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-data\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.267746 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.267705 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-workload-certs\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.267894 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.267866 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-workload-socket\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.268042 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.267949 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-credential-socket\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.268109 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.267969 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istiod-ca-cert\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.268109 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.268074 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-data\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.269844 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.269809 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-envoy\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.270332 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.270296 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-podinfo\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.275233 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.275204 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-token\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.275699 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.275666 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmb6d\" (UniqueName: \"kubernetes.io/projected/e48eac13-a37f-4508-a5c5-b5f27b86abc8-kube-api-access-kmb6d\") pod \"e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.461992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.461881 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:43.618453 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.618426 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp"] Jul 14 01:33:43.620564 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:33:43.620532 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode48eac13_a37f_4508_a5c5_b5f27b86abc8.slice/crio-2563f80da5798b8a42eee5bd7943cb10087395c3d97558a043f5077c88b687f7 WatchSource:0}: Error finding container 2563f80da5798b8a42eee5bd7943cb10087395c3d97558a043f5077c88b687f7: Status 404 returned error can't find the container with id 2563f80da5798b8a42eee5bd7943cb10087395c3d97558a043f5077c88b687f7 Jul 14 01:33:43.622635 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.622603 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:33:43.622733 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.622681 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:33:43.622733 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.622726 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:33:43.753895 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.753794 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" event={"ID":"e48eac13-a37f-4508-a5c5-b5f27b86abc8","Type":"ContainerStarted","Data":"0b193c66b65ae6712319515b4ad4121abccb44108f759a3b9af065e923c8ee16"} Jul 14 01:33:43.753895 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.753858 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" event={"ID":"e48eac13-a37f-4508-a5c5-b5f27b86abc8","Type":"ContainerStarted","Data":"2563f80da5798b8a42eee5bd7943cb10087395c3d97558a043f5077c88b687f7"} Jul 14 01:33:43.773172 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:43.772885 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" podStartSLOduration=0.772868042 podStartE2EDuration="772.868042ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:33:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:33:43.771694599 +0000 UTC m=+2052.674550101" watchObservedRunningTime="2026-07-14 01:33:43.772868042 +0000 UTC m=+2052.675723541" Jul 14 01:33:44.462520 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:44.462476 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:44.468000 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:44.467962 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:44.759518 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:44.759431 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:44.760595 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:44.760574 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:56.831688 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:56.831649 2582 generic.go:356] "Generic (PLEG): container finished" podID="e48eac13-a37f-4508-a5c5-b5f27b86abc8" containerID="0b193c66b65ae6712319515b4ad4121abccb44108f759a3b9af065e923c8ee16" exitCode=0 Jul 14 01:33:56.832175 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:56.831726 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" event={"ID":"e48eac13-a37f-4508-a5c5-b5f27b86abc8","Type":"ContainerDied","Data":"0b193c66b65ae6712319515b4ad4121abccb44108f759a3b9af065e923c8ee16"} Jul 14 01:33:56.832175 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:56.832109 2582 scope.go:117] "RemoveContainer" containerID="0b193c66b65ae6712319515b4ad4121abccb44108f759a3b9af065e923c8ee16" Jul 14 01:33:56.833113 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:56.833088 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:33:56.833202 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:56.833151 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:33:56.833202 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:56.833188 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:33:57.839069 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:57.839034 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" event={"ID":"e48eac13-a37f-4508-a5c5-b5f27b86abc8","Type":"ContainerStarted","Data":"94ffd89fcd06f68c3cd1a9fe65b2dce89e8de4d5370e062ec6432b65ea251b36"} Jul 14 01:33:58.462248 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:58.462198 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:33:58.463517 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:58.463493 2582 patch_prober.go:28] interesting pod/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 01:33:58.463614 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:58.463548 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" podUID="e48eac13-a37f-4508-a5c5-b5f27b86abc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 01:33:59.462652 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:59.462612 2582 patch_prober.go:28] interesting pod/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 01:33:59.463053 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:59.462676 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" podUID="e48eac13-a37f-4508-a5c5-b5f27b86abc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 01:33:59.759739 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:33:59.759645 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:34:00.462679 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:00.462643 2582 patch_prober.go:28] interesting pod/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 01:34:00.463068 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:00.462699 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" podUID="e48eac13-a37f-4508-a5c5-b5f27b86abc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 01:34:01.463271 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:01.463232 2582 patch_prober.go:28] interesting pod/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 01:34:01.463694 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:01.463291 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" podUID="e48eac13-a37f-4508-a5c5-b5f27b86abc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 01:34:02.466568 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:02.466532 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:34:02.467499 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:02.467478 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:34:31.725018 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:31.724990 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:34:31.729872 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:31.729842 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:34:38.975164 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:38.975117 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6"] Jul 14 01:34:38.975846 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:38.975822 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da63730-2466-4411-b76c-b69221b7ec5a" containerName="istio-proxy" Jul 14 01:34:38.978944 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:38.978919 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6" Jul 14 01:34:38.981363 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:38.981298 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 01:34:38.981518 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:38.981496 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zn7l8\"" Jul 14 01:34:38.985812 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:38.985780 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6"] Jul 14 01:34:39.087633 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:39.087586 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74beae2d-99b9-46ac-afe0-f428e7516634-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6\" (UID: \"74beae2d-99b9-46ac-afe0-f428e7516634\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6" Jul 14 01:34:39.188667 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:39.188625 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74beae2d-99b9-46ac-afe0-f428e7516634-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6\" (UID: \"74beae2d-99b9-46ac-afe0-f428e7516634\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6" Jul 14 01:34:39.189292 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:39.189268 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74beae2d-99b9-46ac-afe0-f428e7516634-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6\" (UID: \"74beae2d-99b9-46ac-afe0-f428e7516634\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6" Jul 14 01:34:39.292609 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:39.292512 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6" Jul 14 01:34:39.434656 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:39.434629 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6"] Jul 14 01:34:39.436633 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:34:39.436605 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74beae2d_99b9_46ac_afe0_f428e7516634.slice/crio-28dfafb2d80bf6f1329ff08fdb06a7cb521aeda9f26e6af05e1ee794fe314a4d WatchSource:0}: Error finding container 28dfafb2d80bf6f1329ff08fdb06a7cb521aeda9f26e6af05e1ee794fe314a4d: Status 404 returned error can't find the container with id 28dfafb2d80bf6f1329ff08fdb06a7cb521aeda9f26e6af05e1ee794fe314a4d Jul 14 01:34:40.065590 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:40.065498 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6" event={"ID":"74beae2d-99b9-46ac-afe0-f428e7516634","Type":"ContainerStarted","Data":"87b48e30e61b5dadfff852640c652a2902ce2340ac9c279a92c4904f9f1f643b"} Jul 14 01:34:40.065590 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:40.065552 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6" event={"ID":"74beae2d-99b9-46ac-afe0-f428e7516634","Type":"ContainerStarted","Data":"28dfafb2d80bf6f1329ff08fdb06a7cb521aeda9f26e6af05e1ee794fe314a4d"} Jul 14 01:34:40.082520 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:40.082463 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6" podStartSLOduration=2.082442203 podStartE2EDuration="2.082442203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:34:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:34:40.081375255 +0000 UTC m=+2108.984230757" watchObservedRunningTime="2026-07-14 01:34:40.082442203 +0000 UTC m=+2108.985297704" Jul 14 01:34:41.072124 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:41.072084 2582 generic.go:356] "Generic (PLEG): container finished" podID="74beae2d-99b9-46ac-afe0-f428e7516634" containerID="87b48e30e61b5dadfff852640c652a2902ce2340ac9c279a92c4904f9f1f643b" exitCode=0 Jul 14 01:34:41.072564 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:41.072171 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6" event={"ID":"74beae2d-99b9-46ac-afe0-f428e7516634","Type":"ContainerDied","Data":"87b48e30e61b5dadfff852640c652a2902ce2340ac9c279a92c4904f9f1f643b"} Jul 14 01:34:42.211649 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:42.211625 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6" Jul 14 01:34:42.320591 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:42.320550 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74beae2d-99b9-46ac-afe0-f428e7516634-maas-api-service-ca\") pod \"74beae2d-99b9-46ac-afe0-f428e7516634\" (UID: \"74beae2d-99b9-46ac-afe0-f428e7516634\") " Jul 14 01:34:42.320937 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:42.320911 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74beae2d-99b9-46ac-afe0-f428e7516634-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "74beae2d-99b9-46ac-afe0-f428e7516634" (UID: "74beae2d-99b9-46ac-afe0-f428e7516634"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:34:42.422150 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:42.422115 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74beae2d-99b9-46ac-afe0-f428e7516634-maas-api-service-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:34:43.085508 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:43.085477 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6" Jul 14 01:34:43.085682 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:43.085477 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6" event={"ID":"74beae2d-99b9-46ac-afe0-f428e7516634","Type":"ContainerDied","Data":"28dfafb2d80bf6f1329ff08fdb06a7cb521aeda9f26e6af05e1ee794fe314a4d"} Jul 14 01:34:43.085682 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:34:43.085606 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28dfafb2d80bf6f1329ff08fdb06a7cb521aeda9f26e6af05e1ee794fe314a4d" Jul 14 01:35:31.825475 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:31.825393 2582 scope.go:117] "RemoveContainer" containerID="8e3cb7b5f14fe2977fb72133659df71bad254dbc5536018043173e5df99b3ddc" Jul 14 01:35:46.370265 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:46.370226 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp"] Jul 14 01:35:46.370689 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:46.370532 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" podUID="e48eac13-a37f-4508-a5c5-b5f27b86abc8" containerName="istio-proxy" containerID="cri-o://94ffd89fcd06f68c3cd1a9fe65b2dce89e8de4d5370e062ec6432b65ea251b36" gracePeriod=30 Jul 14 01:35:47.469887 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:47.469845 2582 patch_prober.go:28] interesting pod/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 01:35:47.470274 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:47.469920 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" podUID="e48eac13-a37f-4508-a5c5-b5f27b86abc8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 01:35:51.624911 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.624878 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:35:51.672735 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.672654 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istiod-ca-cert\") pod \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " Jul 14 01:35:51.672735 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.672723 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-workload-socket\") pod \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " Jul 14 01:35:51.672954 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.672744 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-podinfo\") pod \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " Jul 14 01:35:51.672954 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.672767 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-data\") pod \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " Jul 14 01:35:51.672954 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.672790 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-envoy\") pod \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " Jul 14 01:35:51.672954 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.672817 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-credential-socket\") pod \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " Jul 14 01:35:51.672954 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.672844 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmb6d\" (UniqueName: \"kubernetes.io/projected/e48eac13-a37f-4508-a5c5-b5f27b86abc8-kube-api-access-kmb6d\") pod \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " Jul 14 01:35:51.672954 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.672879 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-workload-certs\") pod \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " Jul 14 01:35:51.672954 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.672900 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-token\") pod \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\" (UID: \"e48eac13-a37f-4508-a5c5-b5f27b86abc8\") " Jul 14 01:35:51.673313 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.673127 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e48eac13-a37f-4508-a5c5-b5f27b86abc8" (UID: "e48eac13-a37f-4508-a5c5-b5f27b86abc8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:35:51.673313 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.673150 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e48eac13-a37f-4508-a5c5-b5f27b86abc8" (UID: "e48eac13-a37f-4508-a5c5-b5f27b86abc8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:35:51.673313 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.673206 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-workload-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:35:51.673494 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.673422 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e48eac13-a37f-4508-a5c5-b5f27b86abc8" (UID: "e48eac13-a37f-4508-a5c5-b5f27b86abc8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:35:51.673494 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.673446 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e48eac13-a37f-4508-a5c5-b5f27b86abc8" (UID: "e48eac13-a37f-4508-a5c5-b5f27b86abc8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:35:51.673939 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.673911 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-data" (OuterVolumeSpecName: "istio-data") pod "e48eac13-a37f-4508-a5c5-b5f27b86abc8" (UID: "e48eac13-a37f-4508-a5c5-b5f27b86abc8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:35:51.675463 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.675439 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-token" (OuterVolumeSpecName: "istio-token") pod "e48eac13-a37f-4508-a5c5-b5f27b86abc8" (UID: "e48eac13-a37f-4508-a5c5-b5f27b86abc8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:35:51.675463 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.675443 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e48eac13-a37f-4508-a5c5-b5f27b86abc8" (UID: "e48eac13-a37f-4508-a5c5-b5f27b86abc8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:35:51.675742 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.675719 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e48eac13-a37f-4508-a5c5-b5f27b86abc8" (UID: "e48eac13-a37f-4508-a5c5-b5f27b86abc8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:35:51.676163 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.676141 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e48eac13-a37f-4508-a5c5-b5f27b86abc8-kube-api-access-kmb6d" (OuterVolumeSpecName: "kube-api-access-kmb6d") pod "e48eac13-a37f-4508-a5c5-b5f27b86abc8" (UID: "e48eac13-a37f-4508-a5c5-b5f27b86abc8"). InnerVolumeSpecName "kube-api-access-kmb6d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:35:51.773723 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.773677 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istiod-ca-cert\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:35:51.773723 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.773714 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-podinfo\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:35:51.773723 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.773727 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-data\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:35:51.774038 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.773741 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-envoy\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:35:51.774038 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.773754 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-credential-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:35:51.774038 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.773769 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmb6d\" (UniqueName: \"kubernetes.io/projected/e48eac13-a37f-4508-a5c5-b5f27b86abc8-kube-api-access-kmb6d\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:35:51.774038 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.773784 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e48eac13-a37f-4508-a5c5-b5f27b86abc8-workload-certs\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:35:51.774038 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:51.773902 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e48eac13-a37f-4508-a5c5-b5f27b86abc8-istio-token\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:35:52.447546 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:52.447508 2582 generic.go:356] "Generic (PLEG): container finished" podID="e48eac13-a37f-4508-a5c5-b5f27b86abc8" containerID="94ffd89fcd06f68c3cd1a9fe65b2dce89e8de4d5370e062ec6432b65ea251b36" exitCode=0 Jul 14 01:35:52.447754 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:52.447555 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" Jul 14 01:35:52.447754 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:52.447574 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" event={"ID":"e48eac13-a37f-4508-a5c5-b5f27b86abc8","Type":"ContainerDied","Data":"94ffd89fcd06f68c3cd1a9fe65b2dce89e8de4d5370e062ec6432b65ea251b36"} Jul 14 01:35:52.447754 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:52.447628 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp" event={"ID":"e48eac13-a37f-4508-a5c5-b5f27b86abc8","Type":"ContainerDied","Data":"2563f80da5798b8a42eee5bd7943cb10087395c3d97558a043f5077c88b687f7"} Jul 14 01:35:52.447754 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:52.447657 2582 scope.go:117] "RemoveContainer" containerID="94ffd89fcd06f68c3cd1a9fe65b2dce89e8de4d5370e062ec6432b65ea251b36" Jul 14 01:35:52.456876 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:52.456859 2582 scope.go:117] "RemoveContainer" containerID="0b193c66b65ae6712319515b4ad4121abccb44108f759a3b9af065e923c8ee16" Jul 14 01:35:52.465064 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:52.465046 2582 scope.go:117] "RemoveContainer" containerID="94ffd89fcd06f68c3cd1a9fe65b2dce89e8de4d5370e062ec6432b65ea251b36" Jul 14 01:35:52.465313 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:35:52.465291 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94ffd89fcd06f68c3cd1a9fe65b2dce89e8de4d5370e062ec6432b65ea251b36\": container with ID starting with 94ffd89fcd06f68c3cd1a9fe65b2dce89e8de4d5370e062ec6432b65ea251b36 not found: ID does not exist" containerID="94ffd89fcd06f68c3cd1a9fe65b2dce89e8de4d5370e062ec6432b65ea251b36" Jul 14 01:35:52.465483 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:52.465341 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ffd89fcd06f68c3cd1a9fe65b2dce89e8de4d5370e062ec6432b65ea251b36"} err="failed to get container status \"94ffd89fcd06f68c3cd1a9fe65b2dce89e8de4d5370e062ec6432b65ea251b36\": rpc error: code = NotFound desc = could not find container \"94ffd89fcd06f68c3cd1a9fe65b2dce89e8de4d5370e062ec6432b65ea251b36\": container with ID starting with 94ffd89fcd06f68c3cd1a9fe65b2dce89e8de4d5370e062ec6432b65ea251b36 not found: ID does not exist" Jul 14 01:35:52.465483 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:52.465363 2582 scope.go:117] "RemoveContainer" containerID="0b193c66b65ae6712319515b4ad4121abccb44108f759a3b9af065e923c8ee16" Jul 14 01:35:52.465844 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:35:52.465827 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b193c66b65ae6712319515b4ad4121abccb44108f759a3b9af065e923c8ee16\": container with ID starting with 0b193c66b65ae6712319515b4ad4121abccb44108f759a3b9af065e923c8ee16 not found: ID does not exist" containerID="0b193c66b65ae6712319515b4ad4121abccb44108f759a3b9af065e923c8ee16" Jul 14 01:35:52.465881 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:52.465848 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b193c66b65ae6712319515b4ad4121abccb44108f759a3b9af065e923c8ee16"} err="failed to get container status \"0b193c66b65ae6712319515b4ad4121abccb44108f759a3b9af065e923c8ee16\": rpc error: code = NotFound desc = could not find container \"0b193c66b65ae6712319515b4ad4121abccb44108f759a3b9af065e923c8ee16\": container with ID starting with 0b193c66b65ae6712319515b4ad4121abccb44108f759a3b9af065e923c8ee16 not found: ID does not exist" Jul 14 01:35:52.469221 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:52.469198 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp"] Jul 14 01:35:52.472670 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:52.472649 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-45ee56e2-openshift-default-69c74f77b7-dknvp"] Jul 14 01:35:53.692986 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:35:53.692954 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e48eac13-a37f-4508-a5c5-b5f27b86abc8" path="/var/lib/kubelet/pods/e48eac13-a37f-4508-a5c5-b5f27b86abc8/volumes" Jul 14 01:36:31.845982 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:31.845946 2582 scope.go:117] "RemoveContainer" containerID="9723f327df3f88b369b5a651efb0d2e92409546438a1bdc095a66c911760d72a" Jul 14 01:36:40.790562 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.790528 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d"] Jul 14 01:36:40.791033 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.791013 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e48eac13-a37f-4508-a5c5-b5f27b86abc8" containerName="istio-proxy" Jul 14 01:36:40.791112 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.791036 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e48eac13-a37f-4508-a5c5-b5f27b86abc8" containerName="istio-proxy" Jul 14 01:36:40.791112 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.791053 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74beae2d-99b9-46ac-afe0-f428e7516634" containerName="revoke-keys" Jul 14 01:36:40.791112 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.791062 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="74beae2d-99b9-46ac-afe0-f428e7516634" containerName="revoke-keys" Jul 14 01:36:40.791255 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.791186 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e48eac13-a37f-4508-a5c5-b5f27b86abc8" containerName="istio-proxy" Jul 14 01:36:40.791255 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.791201 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="74beae2d-99b9-46ac-afe0-f428e7516634" containerName="revoke-keys" Jul 14 01:36:40.791255 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.791210 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e48eac13-a37f-4508-a5c5-b5f27b86abc8" containerName="istio-proxy" Jul 14 01:36:40.794508 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.794487 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.797055 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.797039 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c6eb1c-openshift-default-dockercfg-nkg7v\"" Jul 14 01:36:40.806652 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.806627 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d"] Jul 14 01:36:40.815425 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.815398 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29c6d517-b091-4584-8593-8d3c8961ab9a-istiod-ca-cert\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.815551 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.815433 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-envoy\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.815551 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.815452 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-podinfo\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.815661 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.815556 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-workload-certs\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.815661 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.815618 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-data\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.815661 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.815652 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-token\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.815842 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.815725 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-credential-socket\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.815842 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.815777 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-workload-socket\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.815842 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.815805 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqv8r\" (UniqueName: \"kubernetes.io/projected/29c6d517-b091-4584-8593-8d3c8961ab9a-kube-api-access-wqv8r\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.916574 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.916532 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-workload-socket\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.916760 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.916586 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqv8r\" (UniqueName: \"kubernetes.io/projected/29c6d517-b091-4584-8593-8d3c8961ab9a-kube-api-access-wqv8r\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.916760 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.916629 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29c6d517-b091-4584-8593-8d3c8961ab9a-istiod-ca-cert\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.916760 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.916659 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-envoy\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.916760 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.916685 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-podinfo\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.916760 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.916726 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-workload-certs\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.917035 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.917001 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-workload-socket\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.917136 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.917113 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-workload-certs\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.917207 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.916793 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-data\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.917207 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.917172 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-data\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.917207 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.917184 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-token\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.917404 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.917237 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-credential-socket\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.917465 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.917450 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29c6d517-b091-4584-8593-8d3c8961ab9a-istiod-ca-cert\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.917567 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.917551 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-credential-socket\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.919228 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.919193 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-envoy\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.919368 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.919267 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-podinfo\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.929189 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.929159 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-token\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:40.929287 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:40.929194 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqv8r\" (UniqueName: \"kubernetes.io/projected/29c6d517-b091-4584-8593-8d3c8961ab9a-kube-api-access-wqv8r\") pod \"e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:41.105804 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:41.105765 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:41.250848 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:41.250815 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d"] Jul 14 01:36:41.252794 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:36:41.252759 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29c6d517_b091_4584_8593_8d3c8961ab9a.slice/crio-2cb8ddeef9d26bf22ab814e9ee38f605bcedda773d549a02ac91669150195b3c WatchSource:0}: Error finding container 2cb8ddeef9d26bf22ab814e9ee38f605bcedda773d549a02ac91669150195b3c: Status 404 returned error can't find the container with id 2cb8ddeef9d26bf22ab814e9ee38f605bcedda773d549a02ac91669150195b3c Jul 14 01:36:41.254938 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:41.254903 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:36:41.255025 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:41.254970 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:36:41.259386 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:41.255438 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:36:41.706725 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:41.706678 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" event={"ID":"29c6d517-b091-4584-8593-8d3c8961ab9a","Type":"ContainerStarted","Data":"6537d45078db0ab87d149b4beb8b909d0294779a2320cc6d8ee6eace86314070"} Jul 14 01:36:41.706927 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:41.706737 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" event={"ID":"29c6d517-b091-4584-8593-8d3c8961ab9a","Type":"ContainerStarted","Data":"2cb8ddeef9d26bf22ab814e9ee38f605bcedda773d549a02ac91669150195b3c"} Jul 14 01:36:41.726183 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:41.726133 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" podStartSLOduration=1.7261169 podStartE2EDuration="1.7261169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:36:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:36:41.725067564 +0000 UTC m=+2230.627923063" watchObservedRunningTime="2026-07-14 01:36:41.7261169 +0000 UTC m=+2230.628972399" Jul 14 01:36:42.106985 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:42.106947 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:42.111958 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:42.111930 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:42.712373 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:42.712341 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:42.713217 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:42.713196 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:36:58.754538 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:58.754481 2582 patch_prober.go:28] interesting pod/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 14 01:36:58.755056 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:58.754561 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" podUID="29c6d517-b091-4584-8593-8d3c8961ab9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 14 01:36:59.806383 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:59.806352 2582 generic.go:356] "Generic (PLEG): container finished" podID="29c6d517-b091-4584-8593-8d3c8961ab9a" containerID="6537d45078db0ab87d149b4beb8b909d0294779a2320cc6d8ee6eace86314070" exitCode=0 Jul 14 01:36:59.806800 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:59.806425 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" event={"ID":"29c6d517-b091-4584-8593-8d3c8961ab9a","Type":"ContainerDied","Data":"6537d45078db0ab87d149b4beb8b909d0294779a2320cc6d8ee6eace86314070"} Jul 14 01:36:59.806850 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:59.806831 2582 scope.go:117] "RemoveContainer" containerID="6537d45078db0ab87d149b4beb8b909d0294779a2320cc6d8ee6eace86314070" Jul 14 01:36:59.807951 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:59.807925 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:36:59.808071 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:59.808024 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:36:59.808147 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:36:59.808077 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:37:00.812846 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:00.812802 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" event={"ID":"29c6d517-b091-4584-8593-8d3c8961ab9a","Type":"ContainerStarted","Data":"f2a07aee4f9c11ed49da948e39cd3c2b6b669e7e23a9997f3a74a1bfd3b762de"} Jul 14 01:37:01.106493 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:01.106455 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:37:01.106687 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:01.106665 2582 patch_prober.go:28] interesting pod/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 01:37:01.106739 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:01.106719 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" podUID="29c6d517-b091-4584-8593-8d3c8961ab9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 01:37:02.106622 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:02.106582 2582 patch_prober.go:28] interesting pod/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 01:37:02.107032 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:02.106648 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" podUID="29c6d517-b091-4584-8593-8d3c8961ab9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 01:37:03.106600 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:03.106562 2582 patch_prober.go:28] interesting pod/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 01:37:03.106912 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:03.106626 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" podUID="29c6d517-b091-4584-8593-8d3c8961ab9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 01:37:04.106146 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:04.106104 2582 patch_prober.go:28] interesting pod/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 01:37:04.106358 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:04.106166 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" podUID="29c6d517-b091-4584-8593-8d3c8961ab9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 01:37:05.109948 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:05.109910 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:37:05.110400 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:05.110261 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:37:05.111007 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:05.110989 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:37:50.585563 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:50.585522 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc"] Jul 14 01:37:50.586032 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:50.585896 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e48eac13-a37f-4508-a5c5-b5f27b86abc8" containerName="istio-proxy" Jul 14 01:37:50.586032 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:50.585907 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e48eac13-a37f-4508-a5c5-b5f27b86abc8" containerName="istio-proxy" Jul 14 01:37:50.589013 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:50.588996 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc" Jul 14 01:37:50.591672 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:50.591652 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 01:37:50.591777 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:50.591653 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zn7l8\"" Jul 14 01:37:50.597452 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:50.597423 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc"] Jul 14 01:37:50.662912 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:50.662878 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/621d7a17-8b74-41ac-9501-ca4ac2a2ff01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc\" (UID: \"621d7a17-8b74-41ac-9501-ca4ac2a2ff01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc" Jul 14 01:37:50.764007 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:50.763971 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/621d7a17-8b74-41ac-9501-ca4ac2a2ff01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc\" (UID: \"621d7a17-8b74-41ac-9501-ca4ac2a2ff01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc" Jul 14 01:37:50.764704 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:50.764681 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/621d7a17-8b74-41ac-9501-ca4ac2a2ff01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc\" (UID: \"621d7a17-8b74-41ac-9501-ca4ac2a2ff01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc" Jul 14 01:37:50.901308 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:50.901226 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc" Jul 14 01:37:51.037945 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:51.037912 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc"] Jul 14 01:37:51.039620 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:37:51.039587 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod621d7a17_8b74_41ac_9501_ca4ac2a2ff01.slice/crio-8f99648291820bc71cbea9fde6a6e1d3dc0d62566afd6e95fa64590b3527ee4d WatchSource:0}: Error finding container 8f99648291820bc71cbea9fde6a6e1d3dc0d62566afd6e95fa64590b3527ee4d: Status 404 returned error can't find the container with id 8f99648291820bc71cbea9fde6a6e1d3dc0d62566afd6e95fa64590b3527ee4d Jul 14 01:37:51.078082 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:51.078045 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc" event={"ID":"621d7a17-8b74-41ac-9501-ca4ac2a2ff01","Type":"ContainerStarted","Data":"8f99648291820bc71cbea9fde6a6e1d3dc0d62566afd6e95fa64590b3527ee4d"} Jul 14 01:37:52.085741 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:52.085699 2582 generic.go:356] "Generic (PLEG): container finished" podID="621d7a17-8b74-41ac-9501-ca4ac2a2ff01" containerID="6184a0bff89c183e7b163757ff989edebd6472856dd765037dc60a8c4d020bb9" exitCode=0 Jul 14 01:37:52.086179 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:52.085782 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc" event={"ID":"621d7a17-8b74-41ac-9501-ca4ac2a2ff01","Type":"ContainerDied","Data":"6184a0bff89c183e7b163757ff989edebd6472856dd765037dc60a8c4d020bb9"} Jul 14 01:37:53.222010 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:53.221982 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc" Jul 14 01:37:53.388136 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:53.388042 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/621d7a17-8b74-41ac-9501-ca4ac2a2ff01-maas-api-service-ca\") pod \"621d7a17-8b74-41ac-9501-ca4ac2a2ff01\" (UID: \"621d7a17-8b74-41ac-9501-ca4ac2a2ff01\") " Jul 14 01:37:53.388425 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:53.388401 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/621d7a17-8b74-41ac-9501-ca4ac2a2ff01-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "621d7a17-8b74-41ac-9501-ca4ac2a2ff01" (UID: "621d7a17-8b74-41ac-9501-ca4ac2a2ff01"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:37:53.489420 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:53.489382 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/621d7a17-8b74-41ac-9501-ca4ac2a2ff01-maas-api-service-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:37:54.097276 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:54.097239 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc" Jul 14 01:37:54.097493 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:54.097246 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc" event={"ID":"621d7a17-8b74-41ac-9501-ca4ac2a2ff01","Type":"ContainerDied","Data":"8f99648291820bc71cbea9fde6a6e1d3dc0d62566afd6e95fa64590b3527ee4d"} Jul 14 01:37:54.097493 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:37:54.097402 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f99648291820bc71cbea9fde6a6e1d3dc0d62566afd6e95fa64590b3527ee4d" Jul 14 01:38:16.323280 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:16.323241 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d"] Jul 14 01:38:16.323776 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:16.323485 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" podUID="29c6d517-b091-4584-8593-8d3c8961ab9a" containerName="istio-proxy" containerID="cri-o://f2a07aee4f9c11ed49da948e39cd3c2b6b669e7e23a9997f3a74a1bfd3b762de" gracePeriod=30 Jul 14 01:38:16.689907 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:16.689872 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t"] Jul 14 01:38:16.690236 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:16.690224 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="621d7a17-8b74-41ac-9501-ca4ac2a2ff01" containerName="revoke-keys" Jul 14 01:38:16.690287 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:16.690238 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="621d7a17-8b74-41ac-9501-ca4ac2a2ff01" containerName="revoke-keys" Jul 14 01:38:16.690341 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:16.690311 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="621d7a17-8b74-41ac-9501-ca4ac2a2ff01" containerName="revoke-keys" Jul 14 01:38:16.693462 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:16.693445 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t" Jul 14 01:38:16.696163 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:16.696146 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zn7l8\"" Jul 14 01:38:16.696220 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:16.696146 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 01:38:16.699933 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:16.699893 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t"] Jul 14 01:38:16.801648 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:16.801608 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ff4d73-1690-4638-ab39-33ac9e2cefa1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t\" (UID: \"c8ff4d73-1690-4638-ab39-33ac9e2cefa1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t" Jul 14 01:38:16.902095 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:16.902055 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ff4d73-1690-4638-ab39-33ac9e2cefa1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t\" (UID: \"c8ff4d73-1690-4638-ab39-33ac9e2cefa1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t" Jul 14 01:38:16.902721 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:16.902702 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ff4d73-1690-4638-ab39-33ac9e2cefa1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t\" (UID: \"c8ff4d73-1690-4638-ab39-33ac9e2cefa1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t" Jul 14 01:38:17.005773 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:17.005680 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t" Jul 14 01:38:17.141305 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:17.141275 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t"] Jul 14 01:38:17.142879 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:38:17.142854 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8ff4d73_1690_4638_ab39_33ac9e2cefa1.slice/crio-0f50f77fb544f5f0a0cbe9f754447d3628a568e9b3ec9abd6e3dfb4dee26a9d4 WatchSource:0}: Error finding container 0f50f77fb544f5f0a0cbe9f754447d3628a568e9b3ec9abd6e3dfb4dee26a9d4: Status 404 returned error can't find the container with id 0f50f77fb544f5f0a0cbe9f754447d3628a568e9b3ec9abd6e3dfb4dee26a9d4 Jul 14 01:38:17.217489 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:17.217456 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t" event={"ID":"c8ff4d73-1690-4638-ab39-33ac9e2cefa1","Type":"ContainerStarted","Data":"0f50f77fb544f5f0a0cbe9f754447d3628a568e9b3ec9abd6e3dfb4dee26a9d4"} Jul 14 01:38:18.223885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:18.223797 2582 generic.go:356] "Generic (PLEG): container finished" podID="c8ff4d73-1690-4638-ab39-33ac9e2cefa1" containerID="7015548a073d96b6059c9917724b1aa95f0e1b20568990b86efea3bfc408c37d" exitCode=0 Jul 14 01:38:18.223885 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:18.223853 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t" event={"ID":"c8ff4d73-1690-4638-ab39-33ac9e2cefa1","Type":"ContainerDied","Data":"7015548a073d96b6059c9917724b1aa95f0e1b20568990b86efea3bfc408c37d"} Jul 14 01:38:19.359850 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:19.359824 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t" Jul 14 01:38:19.425024 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:19.424990 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ff4d73-1690-4638-ab39-33ac9e2cefa1-maas-api-service-ca\") pod \"c8ff4d73-1690-4638-ab39-33ac9e2cefa1\" (UID: \"c8ff4d73-1690-4638-ab39-33ac9e2cefa1\") " Jul 14 01:38:19.430348 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:19.425832 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8ff4d73-1690-4638-ab39-33ac9e2cefa1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c8ff4d73-1690-4638-ab39-33ac9e2cefa1" (UID: "c8ff4d73-1690-4638-ab39-33ac9e2cefa1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:38:19.526002 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:19.525906 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ff4d73-1690-4638-ab39-33ac9e2cefa1-maas-api-service-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:38:20.112972 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:20.112935 2582 patch_prober.go:28] interesting pod/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 01:38:20.113138 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:20.113000 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" podUID="29c6d517-b091-4584-8593-8d3c8961ab9a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 01:38:20.235733 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:20.235708 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t" Jul 14 01:38:20.235733 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:20.235710 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t" event={"ID":"c8ff4d73-1690-4638-ab39-33ac9e2cefa1","Type":"ContainerDied","Data":"0f50f77fb544f5f0a0cbe9f754447d3628a568e9b3ec9abd6e3dfb4dee26a9d4"} Jul 14 01:38:20.235947 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:20.235760 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f50f77fb544f5f0a0cbe9f754447d3628a568e9b3ec9abd6e3dfb4dee26a9d4" Jul 14 01:38:21.580533 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.580506 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:38:21.645629 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.645593 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqv8r\" (UniqueName: \"kubernetes.io/projected/29c6d517-b091-4584-8593-8d3c8961ab9a-kube-api-access-wqv8r\") pod \"29c6d517-b091-4584-8593-8d3c8961ab9a\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " Jul 14 01:38:21.645831 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.645658 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-data\") pod \"29c6d517-b091-4584-8593-8d3c8961ab9a\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " Jul 14 01:38:21.645831 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.645703 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-podinfo\") pod \"29c6d517-b091-4584-8593-8d3c8961ab9a\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " Jul 14 01:38:21.645831 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.645724 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-workload-certs\") pod \"29c6d517-b091-4584-8593-8d3c8961ab9a\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " Jul 14 01:38:21.645831 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.645762 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-token\") pod \"29c6d517-b091-4584-8593-8d3c8961ab9a\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " Jul 14 01:38:21.645831 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.645802 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-credential-socket\") pod \"29c6d517-b091-4584-8593-8d3c8961ab9a\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " Jul 14 01:38:21.645831 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.645826 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29c6d517-b091-4584-8593-8d3c8961ab9a-istiod-ca-cert\") pod \"29c6d517-b091-4584-8593-8d3c8961ab9a\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " Jul 14 01:38:21.646139 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.645852 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-envoy\") pod \"29c6d517-b091-4584-8593-8d3c8961ab9a\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " Jul 14 01:38:21.646139 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.645895 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-workload-socket\") pod \"29c6d517-b091-4584-8593-8d3c8961ab9a\" (UID: \"29c6d517-b091-4584-8593-8d3c8961ab9a\") " Jul 14 01:38:21.646139 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.646034 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "29c6d517-b091-4584-8593-8d3c8961ab9a" (UID: "29c6d517-b091-4584-8593-8d3c8961ab9a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:38:21.646297 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.646240 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-workload-certs\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:38:21.646385 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.646275 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "29c6d517-b091-4584-8593-8d3c8961ab9a" (UID: "29c6d517-b091-4584-8593-8d3c8961ab9a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:38:21.646441 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.646396 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29c6d517-b091-4584-8593-8d3c8961ab9a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "29c6d517-b091-4584-8593-8d3c8961ab9a" (UID: "29c6d517-b091-4584-8593-8d3c8961ab9a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:38:21.646553 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.646515 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "29c6d517-b091-4584-8593-8d3c8961ab9a" (UID: "29c6d517-b091-4584-8593-8d3c8961ab9a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:38:21.646690 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.646667 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-data" (OuterVolumeSpecName: "istio-data") pod "29c6d517-b091-4584-8593-8d3c8961ab9a" (UID: "29c6d517-b091-4584-8593-8d3c8961ab9a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:38:21.648021 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.647996 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29c6d517-b091-4584-8593-8d3c8961ab9a-kube-api-access-wqv8r" (OuterVolumeSpecName: "kube-api-access-wqv8r") pod "29c6d517-b091-4584-8593-8d3c8961ab9a" (UID: "29c6d517-b091-4584-8593-8d3c8961ab9a"). InnerVolumeSpecName "kube-api-access-wqv8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:38:21.648021 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.648012 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "29c6d517-b091-4584-8593-8d3c8961ab9a" (UID: "29c6d517-b091-4584-8593-8d3c8961ab9a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:38:21.648279 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.648266 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "29c6d517-b091-4584-8593-8d3c8961ab9a" (UID: "29c6d517-b091-4584-8593-8d3c8961ab9a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:38:21.648606 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.648585 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-token" (OuterVolumeSpecName: "istio-token") pod "29c6d517-b091-4584-8593-8d3c8961ab9a" (UID: "29c6d517-b091-4584-8593-8d3c8961ab9a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:38:21.747233 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.747203 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqv8r\" (UniqueName: \"kubernetes.io/projected/29c6d517-b091-4584-8593-8d3c8961ab9a-kube-api-access-wqv8r\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:38:21.747233 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.747230 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-data\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:38:21.747233 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.747239 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-podinfo\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:38:21.747556 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.747253 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-token\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:38:21.747556 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.747267 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-credential-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:38:21.747556 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.747275 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29c6d517-b091-4584-8593-8d3c8961ab9a-istiod-ca-cert\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:38:21.747556 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.747284 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-istio-envoy\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:38:21.747556 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:21.747291 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6d517-b091-4584-8593-8d3c8961ab9a-workload-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:38:22.247913 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:22.247879 2582 generic.go:356] "Generic (PLEG): container finished" podID="29c6d517-b091-4584-8593-8d3c8961ab9a" containerID="f2a07aee4f9c11ed49da948e39cd3c2b6b669e7e23a9997f3a74a1bfd3b762de" exitCode=0 Jul 14 01:38:22.248099 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:22.247927 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" Jul 14 01:38:22.248099 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:22.247938 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" event={"ID":"29c6d517-b091-4584-8593-8d3c8961ab9a","Type":"ContainerDied","Data":"f2a07aee4f9c11ed49da948e39cd3c2b6b669e7e23a9997f3a74a1bfd3b762de"} Jul 14 01:38:22.248099 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:22.247982 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d" event={"ID":"29c6d517-b091-4584-8593-8d3c8961ab9a","Type":"ContainerDied","Data":"2cb8ddeef9d26bf22ab814e9ee38f605bcedda773d549a02ac91669150195b3c"} Jul 14 01:38:22.248099 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:22.248004 2582 scope.go:117] "RemoveContainer" containerID="f2a07aee4f9c11ed49da948e39cd3c2b6b669e7e23a9997f3a74a1bfd3b762de" Jul 14 01:38:22.257584 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:22.257520 2582 scope.go:117] "RemoveContainer" containerID="6537d45078db0ab87d149b4beb8b909d0294779a2320cc6d8ee6eace86314070" Jul 14 01:38:22.266611 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:22.266527 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d"] Jul 14 01:38:22.266690 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:22.266635 2582 scope.go:117] "RemoveContainer" containerID="f2a07aee4f9c11ed49da948e39cd3c2b6b669e7e23a9997f3a74a1bfd3b762de" Jul 14 01:38:22.266934 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:38:22.266912 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2a07aee4f9c11ed49da948e39cd3c2b6b669e7e23a9997f3a74a1bfd3b762de\": container with ID starting with f2a07aee4f9c11ed49da948e39cd3c2b6b669e7e23a9997f3a74a1bfd3b762de not found: ID does not exist" containerID="f2a07aee4f9c11ed49da948e39cd3c2b6b669e7e23a9997f3a74a1bfd3b762de" Jul 14 01:38:22.267008 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:22.266946 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a07aee4f9c11ed49da948e39cd3c2b6b669e7e23a9997f3a74a1bfd3b762de"} err="failed to get container status \"f2a07aee4f9c11ed49da948e39cd3c2b6b669e7e23a9997f3a74a1bfd3b762de\": rpc error: code = NotFound desc = could not find container \"f2a07aee4f9c11ed49da948e39cd3c2b6b669e7e23a9997f3a74a1bfd3b762de\": container with ID starting with f2a07aee4f9c11ed49da948e39cd3c2b6b669e7e23a9997f3a74a1bfd3b762de not found: ID does not exist" Jul 14 01:38:22.267008 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:22.266974 2582 scope.go:117] "RemoveContainer" containerID="6537d45078db0ab87d149b4beb8b909d0294779a2320cc6d8ee6eace86314070" Jul 14 01:38:22.267214 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:38:22.267192 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6537d45078db0ab87d149b4beb8b909d0294779a2320cc6d8ee6eace86314070\": container with ID starting with 6537d45078db0ab87d149b4beb8b909d0294779a2320cc6d8ee6eace86314070 not found: ID does not exist" containerID="6537d45078db0ab87d149b4beb8b909d0294779a2320cc6d8ee6eace86314070" Jul 14 01:38:22.267257 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:22.267221 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6537d45078db0ab87d149b4beb8b909d0294779a2320cc6d8ee6eace86314070"} err="failed to get container status \"6537d45078db0ab87d149b4beb8b909d0294779a2320cc6d8ee6eace86314070\": rpc error: code = NotFound desc = could not find container \"6537d45078db0ab87d149b4beb8b909d0294779a2320cc6d8ee6eace86314070\": container with ID starting with 6537d45078db0ab87d149b4beb8b909d0294779a2320cc6d8ee6eace86314070 not found: ID does not exist" Jul 14 01:38:22.270288 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:22.270265 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c6eb1c-openshift-default-846697f85c-dvc6d"] Jul 14 01:38:23.692858 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:23.692825 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29c6d517-b091-4584-8593-8d3c8961ab9a" path="/var/lib/kubelet/pods/29c6d517-b091-4584-8593-8d3c8961ab9a/volumes" Jul 14 01:38:43.275924 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.275870 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m"] Jul 14 01:38:43.276821 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.276795 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29c6d517-b091-4584-8593-8d3c8961ab9a" containerName="istio-proxy" Jul 14 01:38:43.276821 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.276823 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="29c6d517-b091-4584-8593-8d3c8961ab9a" containerName="istio-proxy" Jul 14 01:38:43.277001 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.276852 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8ff4d73-1690-4638-ab39-33ac9e2cefa1" containerName="revoke-keys" Jul 14 01:38:43.277001 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.276860 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8ff4d73-1690-4638-ab39-33ac9e2cefa1" containerName="revoke-keys" Jul 14 01:38:43.277001 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.276963 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="29c6d517-b091-4584-8593-8d3c8961ab9a" containerName="istio-proxy" Jul 14 01:38:43.277001 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.276979 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8ff4d73-1690-4638-ab39-33ac9e2cefa1" containerName="revoke-keys" Jul 14 01:38:43.277001 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.276994 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="29c6d517-b091-4584-8593-8d3c8961ab9a" containerName="istio-proxy" Jul 14 01:38:43.280400 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.280378 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.282906 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.282883 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ef50e2-openshift-default-dockercfg-88fsd\"" Jul 14 01:38:43.291649 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.291622 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m"] Jul 14 01:38:43.343861 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.343828 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/214de5b9-8099-4565-9467-076729049002-istio-token\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.344054 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.343873 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-workload-socket\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.344054 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.343897 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qbzs\" (UniqueName: \"kubernetes.io/projected/214de5b9-8099-4565-9467-076729049002-kube-api-access-5qbzs\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.344054 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.344004 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-istio-data\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.344054 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.344043 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/214de5b9-8099-4565-9467-076729049002-istio-podinfo\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.344242 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.344068 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-credential-socket\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.344242 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.344150 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-workload-certs\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.344242 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.344181 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-istio-envoy\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.344416 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.344252 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/214de5b9-8099-4565-9467-076729049002-istiod-ca-cert\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.445715 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.445674 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-istio-data\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.445715 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.445712 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/214de5b9-8099-4565-9467-076729049002-istio-podinfo\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.445970 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.445741 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-credential-socket\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.445970 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.445775 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-workload-certs\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.445970 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.445798 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-istio-envoy\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.445970 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.445842 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/214de5b9-8099-4565-9467-076729049002-istiod-ca-cert\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.445970 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.445925 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/214de5b9-8099-4565-9467-076729049002-istio-token\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.445970 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.445965 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-workload-socket\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.446384 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.445987 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qbzs\" (UniqueName: \"kubernetes.io/projected/214de5b9-8099-4565-9467-076729049002-kube-api-access-5qbzs\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.446384 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.446148 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-istio-data\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.446384 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.446163 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-credential-socket\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.446384 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.446223 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-workload-certs\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.446384 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.446365 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-workload-socket\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.447096 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.447071 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/214de5b9-8099-4565-9467-076729049002-istiod-ca-cert\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.448416 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.448386 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-istio-envoy\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.448685 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.448667 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/214de5b9-8099-4565-9467-076729049002-istio-podinfo\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.454063 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.454033 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/214de5b9-8099-4565-9467-076729049002-istio-token\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.454248 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.454229 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qbzs\" (UniqueName: \"kubernetes.io/projected/214de5b9-8099-4565-9467-076729049002-kube-api-access-5qbzs\") pod \"e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.594528 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.594492 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:43.741360 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.741312 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m"] Jul 14 01:38:43.743155 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:38:43.743129 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod214de5b9_8099_4565_9467_076729049002.slice/crio-12c04cc5fcf637841c3f36ea601d3a563d7e0ff55fb07c875dc5a7848263dcfb WatchSource:0}: Error finding container 12c04cc5fcf637841c3f36ea601d3a563d7e0ff55fb07c875dc5a7848263dcfb: Status 404 returned error can't find the container with id 12c04cc5fcf637841c3f36ea601d3a563d7e0ff55fb07c875dc5a7848263dcfb Jul 14 01:38:43.745438 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.745404 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:38:43.745548 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.745484 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:38:43.745548 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:43.745531 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:38:44.365939 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:44.365905 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" event={"ID":"214de5b9-8099-4565-9467-076729049002","Type":"ContainerStarted","Data":"e02d8f2e4aa4494c9a1a71c0ceb1a5e95ff39992ca7b9aee72a30b9d08913dcf"} Jul 14 01:38:44.366365 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:44.365958 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" event={"ID":"214de5b9-8099-4565-9467-076729049002","Type":"ContainerStarted","Data":"12c04cc5fcf637841c3f36ea601d3a563d7e0ff55fb07c875dc5a7848263dcfb"} Jul 14 01:38:44.386714 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:44.386649 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" podStartSLOduration=1.386631357 podStartE2EDuration="1.386631357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:38:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:38:44.384926275 +0000 UTC m=+2353.287781774" watchObservedRunningTime="2026-07-14 01:38:44.386631357 +0000 UTC m=+2353.289486856" Jul 14 01:38:44.595400 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:44.595351 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:44.600464 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:44.600436 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:45.371331 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:45.371302 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:38:45.372217 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:38:45.372193 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:39:00.373533 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:00.373492 2582 patch_prober.go:28] interesting pod/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 14 01:39:00.374021 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:00.373560 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" podUID="214de5b9-8099-4565-9467-076729049002" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 14 01:39:00.460552 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:00.460516 2582 generic.go:356] "Generic (PLEG): container finished" podID="214de5b9-8099-4565-9467-076729049002" containerID="e02d8f2e4aa4494c9a1a71c0ceb1a5e95ff39992ca7b9aee72a30b9d08913dcf" exitCode=0 Jul 14 01:39:00.460753 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:00.460598 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" event={"ID":"214de5b9-8099-4565-9467-076729049002","Type":"ContainerDied","Data":"e02d8f2e4aa4494c9a1a71c0ceb1a5e95ff39992ca7b9aee72a30b9d08913dcf"} Jul 14 01:39:00.461101 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:00.461083 2582 scope.go:117] "RemoveContainer" containerID="e02d8f2e4aa4494c9a1a71c0ceb1a5e95ff39992ca7b9aee72a30b9d08913dcf" Jul 14 01:39:00.462120 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:00.462095 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:39:00.462207 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:00.462162 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:39:00.462207 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:00.462193 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:39:01.471533 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:01.471494 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" event={"ID":"214de5b9-8099-4565-9467-076729049002","Type":"ContainerStarted","Data":"808acec4069e696827434e7418837fc391a02299ebe612e8b8495411b854327b"} Jul 14 01:39:01.595357 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:01.595304 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:39:01.595735 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:01.595708 2582 patch_prober.go:28] interesting pod/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 14 01:39:01.595922 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:01.595898 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" podUID="214de5b9-8099-4565-9467-076729049002" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 14 01:39:02.595931 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:02.595446 2582 patch_prober.go:28] interesting pod/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 14 01:39:02.595931 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:02.595540 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" podUID="214de5b9-8099-4565-9467-076729049002" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 14 01:39:03.595504 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:03.595464 2582 patch_prober.go:28] interesting pod/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 14 01:39:03.595876 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:03.595839 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" podUID="214de5b9-8099-4565-9467-076729049002" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 14 01:39:04.595448 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:04.595404 2582 patch_prober.go:28] interesting pod/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 14 01:39:04.595958 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:04.595472 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" podUID="214de5b9-8099-4565-9467-076729049002" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 14 01:39:05.595875 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:05.595834 2582 patch_prober.go:28] interesting pod/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 14 01:39:05.596298 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:05.595907 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" podUID="214de5b9-8099-4565-9467-076729049002" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 14 01:39:06.599626 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:06.599592 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:39:06.600041 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:06.599965 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:39:06.600829 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:06.600814 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:39:31.756900 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:31.756864 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:39:31.765197 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:31.765172 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:39:32.837204 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:32.837168 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m"] Jul 14 01:39:32.837635 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:32.837402 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" podUID="214de5b9-8099-4565-9467-076729049002" containerName="istio-proxy" containerID="cri-o://808acec4069e696827434e7418837fc391a02299ebe612e8b8495411b854327b" gracePeriod=30 Jul 14 01:39:36.602392 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:36.602347 2582 patch_prober.go:28] interesting pod/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 01:39:36.602774 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:36.602413 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" podUID="214de5b9-8099-4565-9467-076729049002" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 01:39:38.081353 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.081310 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:39:38.273455 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.273354 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/214de5b9-8099-4565-9467-076729049002-istio-podinfo\") pod \"214de5b9-8099-4565-9467-076729049002\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " Jul 14 01:39:38.273455 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.273403 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-credential-socket\") pod \"214de5b9-8099-4565-9467-076729049002\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " Jul 14 01:39:38.273455 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.273425 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/214de5b9-8099-4565-9467-076729049002-istiod-ca-cert\") pod \"214de5b9-8099-4565-9467-076729049002\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " Jul 14 01:39:38.273750 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.273482 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qbzs\" (UniqueName: \"kubernetes.io/projected/214de5b9-8099-4565-9467-076729049002-kube-api-access-5qbzs\") pod \"214de5b9-8099-4565-9467-076729049002\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " Jul 14 01:39:38.273750 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.273499 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-istio-envoy\") pod \"214de5b9-8099-4565-9467-076729049002\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " Jul 14 01:39:38.273750 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.273522 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-istio-data\") pod \"214de5b9-8099-4565-9467-076729049002\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " Jul 14 01:39:38.273750 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.273547 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/214de5b9-8099-4565-9467-076729049002-istio-token\") pod \"214de5b9-8099-4565-9467-076729049002\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " Jul 14 01:39:38.273750 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.273605 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-workload-certs\") pod \"214de5b9-8099-4565-9467-076729049002\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " Jul 14 01:39:38.273750 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.273654 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-workload-socket\") pod \"214de5b9-8099-4565-9467-076729049002\" (UID: \"214de5b9-8099-4565-9467-076729049002\") " Jul 14 01:39:38.273750 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.273702 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "214de5b9-8099-4565-9467-076729049002" (UID: "214de5b9-8099-4565-9467-076729049002"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:39:38.274090 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.273951 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-credential-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:39:38.274090 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.273952 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "214de5b9-8099-4565-9467-076729049002" (UID: "214de5b9-8099-4565-9467-076729049002"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:39:38.274090 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.273974 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/214de5b9-8099-4565-9467-076729049002-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "214de5b9-8099-4565-9467-076729049002" (UID: "214de5b9-8099-4565-9467-076729049002"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:39:38.274250 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.274167 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "214de5b9-8099-4565-9467-076729049002" (UID: "214de5b9-8099-4565-9467-076729049002"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:39:38.274823 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.274793 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-istio-data" (OuterVolumeSpecName: "istio-data") pod "214de5b9-8099-4565-9467-076729049002" (UID: "214de5b9-8099-4565-9467-076729049002"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:39:38.275929 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.275895 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/214de5b9-8099-4565-9467-076729049002-istio-token" (OuterVolumeSpecName: "istio-token") pod "214de5b9-8099-4565-9467-076729049002" (UID: "214de5b9-8099-4565-9467-076729049002"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:39:38.276050 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.275944 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "214de5b9-8099-4565-9467-076729049002" (UID: "214de5b9-8099-4565-9467-076729049002"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:39:38.276362 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.276317 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/214de5b9-8099-4565-9467-076729049002-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "214de5b9-8099-4565-9467-076729049002" (UID: "214de5b9-8099-4565-9467-076729049002"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:39:38.276362 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.276342 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/214de5b9-8099-4565-9467-076729049002-kube-api-access-5qbzs" (OuterVolumeSpecName: "kube-api-access-5qbzs") pod "214de5b9-8099-4565-9467-076729049002" (UID: "214de5b9-8099-4565-9467-076729049002"). InnerVolumeSpecName "kube-api-access-5qbzs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:39:38.374684 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.374644 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-workload-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:39:38.374684 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.374675 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/214de5b9-8099-4565-9467-076729049002-istio-podinfo\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:39:38.374684 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.374684 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/214de5b9-8099-4565-9467-076729049002-istiod-ca-cert\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:39:38.374942 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.374696 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qbzs\" (UniqueName: \"kubernetes.io/projected/214de5b9-8099-4565-9467-076729049002-kube-api-access-5qbzs\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:39:38.374942 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.374706 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-istio-envoy\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:39:38.374942 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.374714 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-istio-data\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:39:38.374942 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.374721 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/214de5b9-8099-4565-9467-076729049002-istio-token\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:39:38.374942 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.374730 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/214de5b9-8099-4565-9467-076729049002-workload-certs\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:39:38.699696 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.699663 2582 generic.go:356] "Generic (PLEG): container finished" podID="214de5b9-8099-4565-9467-076729049002" containerID="808acec4069e696827434e7418837fc391a02299ebe612e8b8495411b854327b" exitCode=0 Jul 14 01:39:38.699876 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.699722 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" Jul 14 01:39:38.699876 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.699744 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" event={"ID":"214de5b9-8099-4565-9467-076729049002","Type":"ContainerDied","Data":"808acec4069e696827434e7418837fc391a02299ebe612e8b8495411b854327b"} Jul 14 01:39:38.699876 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.699803 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m" event={"ID":"214de5b9-8099-4565-9467-076729049002","Type":"ContainerDied","Data":"12c04cc5fcf637841c3f36ea601d3a563d7e0ff55fb07c875dc5a7848263dcfb"} Jul 14 01:39:38.699876 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.699825 2582 scope.go:117] "RemoveContainer" containerID="808acec4069e696827434e7418837fc391a02299ebe612e8b8495411b854327b" Jul 14 01:39:38.709777 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.709752 2582 scope.go:117] "RemoveContainer" containerID="e02d8f2e4aa4494c9a1a71c0ceb1a5e95ff39992ca7b9aee72a30b9d08913dcf" Jul 14 01:39:38.718176 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.718156 2582 scope.go:117] "RemoveContainer" containerID="808acec4069e696827434e7418837fc391a02299ebe612e8b8495411b854327b" Jul 14 01:39:38.718476 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:39:38.718455 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"808acec4069e696827434e7418837fc391a02299ebe612e8b8495411b854327b\": container with ID starting with 808acec4069e696827434e7418837fc391a02299ebe612e8b8495411b854327b not found: ID does not exist" containerID="808acec4069e696827434e7418837fc391a02299ebe612e8b8495411b854327b" Jul 14 01:39:38.718526 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.718487 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"808acec4069e696827434e7418837fc391a02299ebe612e8b8495411b854327b"} err="failed to get container status \"808acec4069e696827434e7418837fc391a02299ebe612e8b8495411b854327b\": rpc error: code = NotFound desc = could not find container \"808acec4069e696827434e7418837fc391a02299ebe612e8b8495411b854327b\": container with ID starting with 808acec4069e696827434e7418837fc391a02299ebe612e8b8495411b854327b not found: ID does not exist" Jul 14 01:39:38.718526 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.718506 2582 scope.go:117] "RemoveContainer" containerID="e02d8f2e4aa4494c9a1a71c0ceb1a5e95ff39992ca7b9aee72a30b9d08913dcf" Jul 14 01:39:38.718754 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:39:38.718732 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e02d8f2e4aa4494c9a1a71c0ceb1a5e95ff39992ca7b9aee72a30b9d08913dcf\": container with ID starting with e02d8f2e4aa4494c9a1a71c0ceb1a5e95ff39992ca7b9aee72a30b9d08913dcf not found: ID does not exist" containerID="e02d8f2e4aa4494c9a1a71c0ceb1a5e95ff39992ca7b9aee72a30b9d08913dcf" Jul 14 01:39:38.718793 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.718761 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e02d8f2e4aa4494c9a1a71c0ceb1a5e95ff39992ca7b9aee72a30b9d08913dcf"} err="failed to get container status \"e02d8f2e4aa4494c9a1a71c0ceb1a5e95ff39992ca7b9aee72a30b9d08913dcf\": rpc error: code = NotFound desc = could not find container \"e02d8f2e4aa4494c9a1a71c0ceb1a5e95ff39992ca7b9aee72a30b9d08913dcf\": container with ID starting with e02d8f2e4aa4494c9a1a71c0ceb1a5e95ff39992ca7b9aee72a30b9d08913dcf not found: ID does not exist" Jul 14 01:39:38.722302 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.722278 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m"] Jul 14 01:39:38.728796 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:38.728769 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ef50e2-openshift-default-7f66cf84d5-cl44m"] Jul 14 01:39:39.693707 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:39.693677 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="214de5b9-8099-4565-9467-076729049002" path="/var/lib/kubelet/pods/214de5b9-8099-4565-9467-076729049002/volumes" Jul 14 01:39:59.846611 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.846575 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s"] Jul 14 01:39:59.847152 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.847132 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29c6d517-b091-4584-8593-8d3c8961ab9a" containerName="istio-proxy" Jul 14 01:39:59.847152 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.847152 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="29c6d517-b091-4584-8593-8d3c8961ab9a" containerName="istio-proxy" Jul 14 01:39:59.847378 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.847167 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="214de5b9-8099-4565-9467-076729049002" containerName="istio-proxy" Jul 14 01:39:59.847378 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.847176 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="214de5b9-8099-4565-9467-076729049002" containerName="istio-proxy" Jul 14 01:39:59.847378 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.847207 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="214de5b9-8099-4565-9467-076729049002" containerName="istio-proxy" Jul 14 01:39:59.847378 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.847216 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="214de5b9-8099-4565-9467-076729049002" containerName="istio-proxy" Jul 14 01:39:59.847378 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.847294 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="214de5b9-8099-4565-9467-076729049002" containerName="istio-proxy" Jul 14 01:39:59.847378 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.847310 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="214de5b9-8099-4565-9467-076729049002" containerName="istio-proxy" Jul 14 01:39:59.852173 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.852152 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:39:59.854621 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.854603 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-c6543c-openshift-default-dockercfg-cmlhz\"" Jul 14 01:39:59.861875 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.861849 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s"] Jul 14 01:39:59.971511 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.971462 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-workload-certs\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:39:59.971696 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.971541 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4231cd7-cf2c-46c1-ac01-a490652861e1-istiod-ca-cert\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:39:59.971696 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.971572 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-envoy\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:39:59.971696 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.971610 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-workload-socket\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:39:59.971696 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.971656 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22vdk\" (UniqueName: \"kubernetes.io/projected/c4231cd7-cf2c-46c1-ac01-a490652861e1-kube-api-access-22vdk\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:39:59.971696 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.971694 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-podinfo\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:39:59.971971 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.971710 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-token\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:39:59.971971 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.971751 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-data\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:39:59.971971 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:39:59.971800 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-credential-socket\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.073231 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.073186 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-workload-certs\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.073448 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.073256 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4231cd7-cf2c-46c1-ac01-a490652861e1-istiod-ca-cert\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.073448 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.073290 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-envoy\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.073448 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.073360 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-workload-socket\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.073448 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.073387 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22vdk\" (UniqueName: \"kubernetes.io/projected/c4231cd7-cf2c-46c1-ac01-a490652861e1-kube-api-access-22vdk\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.073448 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.073409 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-podinfo\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.073448 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.073427 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-token\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.073794 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.073736 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-workload-certs\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.073794 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.073771 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-workload-socket\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.073975 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.073854 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-data\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.073975 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.073917 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-credential-socket\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.074182 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.074130 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4231cd7-cf2c-46c1-ac01-a490652861e1-istiod-ca-cert\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.074182 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.074145 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-data\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.074438 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.074413 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-credential-socket\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.075966 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.075944 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-envoy\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.076111 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.076094 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-podinfo\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.081018 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.080998 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-token\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.081302 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.081275 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22vdk\" (UniqueName: \"kubernetes.io/projected/c4231cd7-cf2c-46c1-ac01-a490652861e1-kube-api-access-22vdk\") pod \"e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.164236 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.164146 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:00.307978 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.307951 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s"] Jul 14 01:40:00.310173 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:40:00.310150 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4231cd7_cf2c_46c1_ac01_a490652861e1.slice/crio-ec3576118354e9cde72d163ba1aebdae382bf78ab90332cc09096f8132329cd1 WatchSource:0}: Error finding container ec3576118354e9cde72d163ba1aebdae382bf78ab90332cc09096f8132329cd1: Status 404 returned error can't find the container with id ec3576118354e9cde72d163ba1aebdae382bf78ab90332cc09096f8132329cd1 Jul 14 01:40:00.312376 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.312342 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:40:00.312461 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.312408 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:40:00.312461 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.312435 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:40:00.815642 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.815606 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" event={"ID":"c4231cd7-cf2c-46c1-ac01-a490652861e1","Type":"ContainerStarted","Data":"de035763c21a7993804c18a482f47119fac517a6949ec9e05f987e416a66f017"} Jul 14 01:40:00.815807 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.815660 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" event={"ID":"c4231cd7-cf2c-46c1-ac01-a490652861e1","Type":"ContainerStarted","Data":"ec3576118354e9cde72d163ba1aebdae382bf78ab90332cc09096f8132329cd1"} Jul 14 01:40:00.836391 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:00.836332 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" podStartSLOduration=1.8363029 podStartE2EDuration="1.8363029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:39:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:40:00.83375451 +0000 UTC m=+2429.736610008" watchObservedRunningTime="2026-07-14 01:40:00.8363029 +0000 UTC m=+2429.739158399" Jul 14 01:40:01.164760 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:01.164716 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:01.169764 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:01.169743 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:01.820721 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:01.820684 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:01.821623 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:01.821599 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:15.901708 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:15.901672 2582 generic.go:356] "Generic (PLEG): container finished" podID="c4231cd7-cf2c-46c1-ac01-a490652861e1" containerID="de035763c21a7993804c18a482f47119fac517a6949ec9e05f987e416a66f017" exitCode=0 Jul 14 01:40:15.902211 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:15.901758 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" event={"ID":"c4231cd7-cf2c-46c1-ac01-a490652861e1","Type":"ContainerDied","Data":"de035763c21a7993804c18a482f47119fac517a6949ec9e05f987e416a66f017"} Jul 14 01:40:15.902294 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:15.902274 2582 scope.go:117] "RemoveContainer" containerID="de035763c21a7993804c18a482f47119fac517a6949ec9e05f987e416a66f017" Jul 14 01:40:15.903382 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:15.903352 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:40:15.903489 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:15.903422 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:40:15.903489 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:15.903455 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 01:40:16.821201 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:16.821160 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:16.910610 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:16.910579 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" event={"ID":"c4231cd7-cf2c-46c1-ac01-a490652861e1","Type":"ContainerStarted","Data":"b1d46b94257d78a33beec05d9671a8338f40bc5729b7c998c357532a5d739285"} Jul 14 01:40:17.164832 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:17.164797 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:17.166055 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:17.166026 2582 patch_prober.go:28] interesting pod/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 14 01:40:17.166179 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:17.166077 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" podUID="c4231cd7-cf2c-46c1-ac01-a490652861e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 14 01:40:18.165231 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:18.165186 2582 patch_prober.go:28] interesting pod/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 14 01:40:18.165676 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:18.165241 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" podUID="c4231cd7-cf2c-46c1-ac01-a490652861e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 14 01:40:19.165180 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:19.165142 2582 patch_prober.go:28] interesting pod/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 14 01:40:19.165395 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:19.165216 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" podUID="c4231cd7-cf2c-46c1-ac01-a490652861e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 14 01:40:20.164574 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:20.164534 2582 patch_prober.go:28] interesting pod/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 14 01:40:20.164752 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:20.164596 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" podUID="c4231cd7-cf2c-46c1-ac01-a490652861e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 14 01:40:21.168408 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:21.168374 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:21.168800 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:21.168648 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:40:21.169243 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:40:21.169225 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:44:31.789098 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:44:31.789016 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:44:31.798758 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:44:31.798732 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:45:00.134022 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.133984 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c6543c-29733225-xz9zl"] Jul 14 01:45:00.137101 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.137083 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a21fee-29733225-lwz4b"] Jul 14 01:45:00.137226 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.137208 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c6543c-29733225-xz9zl" Jul 14 01:45:00.139458 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.139435 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zn7l8\"" Jul 14 01:45:00.140293 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.140268 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733225-4zxpl"] Jul 14 01:45:00.140475 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.140454 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a21fee-29733225-lwz4b" Jul 14 01:45:00.143935 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.143915 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c6543c-29733225-xz9zl"] Jul 14 01:45:00.144041 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.143981 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-4zxpl" Jul 14 01:45:00.149360 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.149156 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a21fee-29733225-lwz4b"] Jul 14 01:45:00.151372 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.151354 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c6543c-29733225-xz9zl" Jul 14 01:45:00.161856 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.161829 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a21fee-29733225-lwz4b" Jul 14 01:45:00.163570 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.163541 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733225-4zxpl"] Jul 14 01:45:00.204976 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.204937 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-4zxpl" Jul 14 01:45:00.324157 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.324127 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c6543c-29733225-xz9zl"] Jul 14 01:45:00.324801 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:45:00.324773 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode057386f_526a_4091_81ce_cb3fcdc95f07.slice/crio-f2b83b230ad675eb9125d1416a734707c135a4ea955b74be71872f82f732cbb9 WatchSource:0}: Error finding container f2b83b230ad675eb9125d1416a734707c135a4ea955b74be71872f82f732cbb9: Status 404 returned error can't find the container with id f2b83b230ad675eb9125d1416a734707c135a4ea955b74be71872f82f732cbb9 Jul 14 01:45:00.376799 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.376771 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733225-4zxpl"] Jul 14 01:45:00.378209 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:45:00.378162 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd478d0f_2e83_453f_a61b_be602a5d2416.slice/crio-ceaa6dbf213f2003fbcdef08c5dd586cb79b304e2bbe927f0b5062f4d2006868 WatchSource:0}: Error finding container ceaa6dbf213f2003fbcdef08c5dd586cb79b304e2bbe927f0b5062f4d2006868: Status 404 returned error can't find the container with id ceaa6dbf213f2003fbcdef08c5dd586cb79b304e2bbe927f0b5062f4d2006868 Jul 14 01:45:00.416017 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.415986 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c6543c-29733225-xz9zl" event={"ID":"e057386f-526a-4091-81ce-cb3fcdc95f07","Type":"ContainerStarted","Data":"f2b83b230ad675eb9125d1416a734707c135a4ea955b74be71872f82f732cbb9"} Jul 14 01:45:00.417134 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.417114 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-4zxpl" event={"ID":"bd478d0f-2e83-453f-a61b-be602a5d2416","Type":"ContainerStarted","Data":"ceaa6dbf213f2003fbcdef08c5dd586cb79b304e2bbe927f0b5062f4d2006868"} Jul 14 01:45:00.557562 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:00.557535 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a21fee-29733225-lwz4b"] Jul 14 01:45:00.558027 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:45:00.558006 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18e3359d_9caa_4fd0_9a12_21e2722154c6.slice/crio-20f1994b9da8403abac38f28e33075a783bfff901bd21053920cb241aaaaedae WatchSource:0}: Error finding container 20f1994b9da8403abac38f28e33075a783bfff901bd21053920cb241aaaaedae: Status 404 returned error can't find the container with id 20f1994b9da8403abac38f28e33075a783bfff901bd21053920cb241aaaaedae Jul 14 01:45:01.429658 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:01.429612 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c6543c-29733225-xz9zl" event={"ID":"e057386f-526a-4091-81ce-cb3fcdc95f07","Type":"ContainerStarted","Data":"8885e9eb3c4058edc50e943fcc19fe31de582a59aef9e18898dfc537702ad6ee"} Jul 14 01:45:01.431103 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:01.431076 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-4zxpl" event={"ID":"bd478d0f-2e83-453f-a61b-be602a5d2416","Type":"ContainerStarted","Data":"fc2165a0ddf77f0218f9606d1385e9bdc0de45bf0b6d1dffa06949dd894d0c1e"} Jul 14 01:45:01.432464 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:01.432442 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a21fee-29733225-lwz4b" event={"ID":"18e3359d-9caa-4fd0-9a12-21e2722154c6","Type":"ContainerStarted","Data":"6022430823b23b44a669d70e301a6d8b4f0521d341fecd1b61a5a0286d8c63f6"} Jul 14 01:45:01.432553 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:01.432486 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a21fee-29733225-lwz4b" event={"ID":"18e3359d-9caa-4fd0-9a12-21e2722154c6","Type":"ContainerStarted","Data":"20f1994b9da8403abac38f28e33075a783bfff901bd21053920cb241aaaaedae"} Jul 14 01:45:01.445311 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:01.445260 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c6543c-29733225-xz9zl" podStartSLOduration=1.445244002 podStartE2EDuration="1.445244002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:45:01.443621707 +0000 UTC m=+2730.346477218" watchObservedRunningTime="2026-07-14 01:45:01.445244002 +0000 UTC m=+2730.348099501" Jul 14 01:45:01.458007 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:01.457954 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-4zxpl" podStartSLOduration=1.457936866 podStartE2EDuration="1.457936866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:45:01.456456293 +0000 UTC m=+2730.359311805" watchObservedRunningTime="2026-07-14 01:45:01.457936866 +0000 UTC m=+2730.360792364" Jul 14 01:45:01.469836 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:01.469791 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a21fee-29733225-lwz4b" podStartSLOduration=1.469776537 podStartE2EDuration="1.469776537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:45:01.468430923 +0000 UTC m=+2730.371286427" watchObservedRunningTime="2026-07-14 01:45:01.469776537 +0000 UTC m=+2730.372632065" Jul 14 01:45:02.440097 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:02.440061 2582 generic.go:356] "Generic (PLEG): container finished" podID="18e3359d-9caa-4fd0-9a12-21e2722154c6" containerID="6022430823b23b44a669d70e301a6d8b4f0521d341fecd1b61a5a0286d8c63f6" exitCode=0 Jul 14 01:45:02.440575 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:02.440155 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a21fee-29733225-lwz4b" event={"ID":"18e3359d-9caa-4fd0-9a12-21e2722154c6","Type":"ContainerDied","Data":"6022430823b23b44a669d70e301a6d8b4f0521d341fecd1b61a5a0286d8c63f6"} Jul 14 01:45:02.441683 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:02.441660 2582 generic.go:356] "Generic (PLEG): container finished" podID="e057386f-526a-4091-81ce-cb3fcdc95f07" containerID="8885e9eb3c4058edc50e943fcc19fe31de582a59aef9e18898dfc537702ad6ee" exitCode=0 Jul 14 01:45:02.441785 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:02.441746 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c6543c-29733225-xz9zl" event={"ID":"e057386f-526a-4091-81ce-cb3fcdc95f07","Type":"ContainerDied","Data":"8885e9eb3c4058edc50e943fcc19fe31de582a59aef9e18898dfc537702ad6ee"} Jul 14 01:45:02.443075 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:02.443054 2582 generic.go:356] "Generic (PLEG): container finished" podID="bd478d0f-2e83-453f-a61b-be602a5d2416" containerID="fc2165a0ddf77f0218f9606d1385e9bdc0de45bf0b6d1dffa06949dd894d0c1e" exitCode=0 Jul 14 01:45:02.443158 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:02.443108 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-4zxpl" event={"ID":"bd478d0f-2e83-453f-a61b-be602a5d2416","Type":"ContainerDied","Data":"fc2165a0ddf77f0218f9606d1385e9bdc0de45bf0b6d1dffa06949dd894d0c1e"} Jul 14 01:45:03.598087 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:03.598058 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a21fee-29733225-lwz4b" Jul 14 01:45:03.637383 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:03.637361 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-4zxpl" Jul 14 01:45:03.640413 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:03.640393 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c6543c-29733225-xz9zl" Jul 14 01:45:04.457291 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:04.457259 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a21fee-29733225-lwz4b" Jul 14 01:45:04.457500 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:04.457298 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a21fee-29733225-lwz4b" event={"ID":"18e3359d-9caa-4fd0-9a12-21e2722154c6","Type":"ContainerDied","Data":"20f1994b9da8403abac38f28e33075a783bfff901bd21053920cb241aaaaedae"} Jul 14 01:45:04.457500 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:04.457379 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20f1994b9da8403abac38f28e33075a783bfff901bd21053920cb241aaaaedae" Jul 14 01:45:04.458587 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:04.458562 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c6543c-29733225-xz9zl" Jul 14 01:45:04.458587 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:04.458580 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c6543c-29733225-xz9zl" event={"ID":"e057386f-526a-4091-81ce-cb3fcdc95f07","Type":"ContainerDied","Data":"f2b83b230ad675eb9125d1416a734707c135a4ea955b74be71872f82f732cbb9"} Jul 14 01:45:04.458796 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:04.458622 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2b83b230ad675eb9125d1416a734707c135a4ea955b74be71872f82f732cbb9" Jul 14 01:45:04.460024 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:04.460006 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-4zxpl" Jul 14 01:45:04.460110 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:04.460032 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733225-4zxpl" event={"ID":"bd478d0f-2e83-453f-a61b-be602a5d2416","Type":"ContainerDied","Data":"ceaa6dbf213f2003fbcdef08c5dd586cb79b304e2bbe927f0b5062f4d2006868"} Jul 14 01:45:04.460110 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:45:04.460063 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ceaa6dbf213f2003fbcdef08c5dd586cb79b304e2bbe927f0b5062f4d2006868" Jul 14 01:46:08.611954 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.611915 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7366-140187614777520"] Jul 14 01:46:08.612459 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.612301 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e057386f-526a-4091-81ce-cb3fcdc95f07" containerName="cleanup" Jul 14 01:46:08.612459 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.612312 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e057386f-526a-4091-81ce-cb3fcdc95f07" containerName="cleanup" Jul 14 01:46:08.612459 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.612344 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd478d0f-2e83-453f-a61b-be602a5d2416" containerName="cleanup" Jul 14 01:46:08.612459 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.612351 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd478d0f-2e83-453f-a61b-be602a5d2416" containerName="cleanup" Jul 14 01:46:08.612459 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.612368 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18e3359d-9caa-4fd0-9a12-21e2722154c6" containerName="cleanup" Jul 14 01:46:08.612459 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.612374 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e3359d-9caa-4fd0-9a12-21e2722154c6" containerName="cleanup" Jul 14 01:46:08.612459 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.612429 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd478d0f-2e83-453f-a61b-be602a5d2416" containerName="cleanup" Jul 14 01:46:08.612459 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.612437 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="18e3359d-9caa-4fd0-9a12-21e2722154c6" containerName="cleanup" Jul 14 01:46:08.612459 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.612444 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e057386f-526a-4091-81ce-cb3fcdc95f07" containerName="cleanup" Jul 14 01:46:08.615634 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.615604 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7366-140187614777520" Jul 14 01:46:08.618246 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.618225 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-2vgml\"" Jul 14 01:46:08.618771 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.618747 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7366-140187614777520"] Jul 14 01:46:08.664756 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.664708 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w89mw\" (UniqueName: \"kubernetes.io/projected/e83eb190-dc5c-4302-9f48-bfa2fdfaf755-kube-api-access-w89mw\") pod \"test-curl-7366-140187614777520\" (UID: \"e83eb190-dc5c-4302-9f48-bfa2fdfaf755\") " pod="openshift-ingress/test-curl-7366-140187614777520" Jul 14 01:46:08.765804 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.765765 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w89mw\" (UniqueName: \"kubernetes.io/projected/e83eb190-dc5c-4302-9f48-bfa2fdfaf755-kube-api-access-w89mw\") pod \"test-curl-7366-140187614777520\" (UID: \"e83eb190-dc5c-4302-9f48-bfa2fdfaf755\") " pod="openshift-ingress/test-curl-7366-140187614777520" Jul 14 01:46:08.774016 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.773983 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w89mw\" (UniqueName: \"kubernetes.io/projected/e83eb190-dc5c-4302-9f48-bfa2fdfaf755-kube-api-access-w89mw\") pod \"test-curl-7366-140187614777520\" (UID: \"e83eb190-dc5c-4302-9f48-bfa2fdfaf755\") " pod="openshift-ingress/test-curl-7366-140187614777520" Jul 14 01:46:08.925938 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:08.925843 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7366-140187614777520" Jul 14 01:46:09.062701 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:09.062671 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7366-140187614777520"] Jul 14 01:46:09.064489 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:46:09.064460 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode83eb190_dc5c_4302_9f48_bfa2fdfaf755.slice/crio-bb6d1d7049f66661c36af998da783c1579edde5942b02374ff362805b2a514e1 WatchSource:0}: Error finding container bb6d1d7049f66661c36af998da783c1579edde5942b02374ff362805b2a514e1: Status 404 returned error can't find the container with id bb6d1d7049f66661c36af998da783c1579edde5942b02374ff362805b2a514e1 Jul 14 01:46:09.065807 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:09.065789 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 01:46:09.829100 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:09.829063 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7366-140187614777520" event={"ID":"e83eb190-dc5c-4302-9f48-bfa2fdfaf755","Type":"ContainerStarted","Data":"bb6d1d7049f66661c36af998da783c1579edde5942b02374ff362805b2a514e1"} Jul 14 01:46:10.835851 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:10.835816 2582 generic.go:356] "Generic (PLEG): container finished" podID="e83eb190-dc5c-4302-9f48-bfa2fdfaf755" containerID="2378fb58a46d30d31fc73b53bb85d7ce65a1bf70b037b6825e446b8af3a17a43" exitCode=0 Jul 14 01:46:10.836286 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:10.835904 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7366-140187614777520" event={"ID":"e83eb190-dc5c-4302-9f48-bfa2fdfaf755","Type":"ContainerDied","Data":"2378fb58a46d30d31fc73b53bb85d7ce65a1bf70b037b6825e446b8af3a17a43"} Jul 14 01:46:11.984594 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:11.984563 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7366-140187614777520" Jul 14 01:46:12.019542 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:12.019517 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7366-140187614777520_e83eb190-dc5c-4302-9f48-bfa2fdfaf755/test-curl-7366-140187614777520/0.log" Jul 14 01:46:12.041562 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:12.041527 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7366-140187614777520"] Jul 14 01:46:12.046304 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:12.045098 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7366-140187614777520"] Jul 14 01:46:12.097018 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:12.096937 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w89mw\" (UniqueName: \"kubernetes.io/projected/e83eb190-dc5c-4302-9f48-bfa2fdfaf755-kube-api-access-w89mw\") pod \"e83eb190-dc5c-4302-9f48-bfa2fdfaf755\" (UID: \"e83eb190-dc5c-4302-9f48-bfa2fdfaf755\") " Jul 14 01:46:12.099168 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:12.099136 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e83eb190-dc5c-4302-9f48-bfa2fdfaf755-kube-api-access-w89mw" (OuterVolumeSpecName: "kube-api-access-w89mw") pod "e83eb190-dc5c-4302-9f48-bfa2fdfaf755" (UID: "e83eb190-dc5c-4302-9f48-bfa2fdfaf755"). InnerVolumeSpecName "kube-api-access-w89mw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:46:12.197829 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:12.197792 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w89mw\" (UniqueName: \"kubernetes.io/projected/e83eb190-dc5c-4302-9f48-bfa2fdfaf755-kube-api-access-w89mw\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:46:12.849023 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:12.848997 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7366-140187614777520" Jul 14 01:46:12.849211 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:12.849046 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb6d1d7049f66661c36af998da783c1579edde5942b02374ff362805b2a514e1" Jul 14 01:46:12.859730 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:46:12.859701 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7366-140187614777520\" is forbidden: User \"system:node:ip-10-0-143-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-159.ec2.internal' and this object" podUID="e83eb190-dc5c-4302-9f48-bfa2fdfaf755" pod="openshift-ingress/test-curl-7366-140187614777520" Jul 14 01:46:13.692748 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:13.692715 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e83eb190-dc5c-4302-9f48-bfa2fdfaf755" path="/var/lib/kubelet/pods/e83eb190-dc5c-4302-9f48-bfa2fdfaf755/volumes" Jul 14 01:46:17.869015 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:17.868979 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7366-140187616088112"] Jul 14 01:46:17.869519 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:17.869381 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e83eb190-dc5c-4302-9f48-bfa2fdfaf755" containerName="test-curl-7366-140187614777520" Jul 14 01:46:17.869519 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:17.869394 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e83eb190-dc5c-4302-9f48-bfa2fdfaf755" containerName="test-curl-7366-140187614777520" Jul 14 01:46:17.869519 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:17.869444 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e83eb190-dc5c-4302-9f48-bfa2fdfaf755" containerName="test-curl-7366-140187614777520" Jul 14 01:46:17.872725 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:17.872703 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7366-140187616088112" Jul 14 01:46:17.877062 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:17.877037 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-2vgml\"" Jul 14 01:46:17.878448 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:17.878420 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7366-140187616088112"] Jul 14 01:46:17.949463 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:17.949422 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqdk8\" (UniqueName: \"kubernetes.io/projected/5664bd90-a736-4150-b744-fec32ec8dc2f-kube-api-access-pqdk8\") pod \"test-curl-7366-140187616088112\" (UID: \"5664bd90-a736-4150-b744-fec32ec8dc2f\") " pod="openshift-ingress/test-curl-7366-140187616088112" Jul 14 01:46:18.050700 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:18.050653 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqdk8\" (UniqueName: \"kubernetes.io/projected/5664bd90-a736-4150-b744-fec32ec8dc2f-kube-api-access-pqdk8\") pod \"test-curl-7366-140187616088112\" (UID: \"5664bd90-a736-4150-b744-fec32ec8dc2f\") " pod="openshift-ingress/test-curl-7366-140187616088112" Jul 14 01:46:18.059193 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:18.059164 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqdk8\" (UniqueName: \"kubernetes.io/projected/5664bd90-a736-4150-b744-fec32ec8dc2f-kube-api-access-pqdk8\") pod \"test-curl-7366-140187616088112\" (UID: \"5664bd90-a736-4150-b744-fec32ec8dc2f\") " pod="openshift-ingress/test-curl-7366-140187616088112" Jul 14 01:46:18.184394 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:18.184276 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7366-140187616088112" Jul 14 01:46:18.326679 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:18.326648 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7366-140187616088112"] Jul 14 01:46:18.328801 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:46:18.328770 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5664bd90_a736_4150_b744_fec32ec8dc2f.slice/crio-1794ab849af3750b0c5480b3396f52306d3f883eb889a28632cc816f507578ef WatchSource:0}: Error finding container 1794ab849af3750b0c5480b3396f52306d3f883eb889a28632cc816f507578ef: Status 404 returned error can't find the container with id 1794ab849af3750b0c5480b3396f52306d3f883eb889a28632cc816f507578ef Jul 14 01:46:18.884646 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:18.884612 2582 generic.go:356] "Generic (PLEG): container finished" podID="5664bd90-a736-4150-b744-fec32ec8dc2f" containerID="9bbf42fb9fbe68d2d4234b03c27ad81ed66669e518a331fc830634abc80ee345" exitCode=0 Jul 14 01:46:18.885095 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:18.884706 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7366-140187616088112" event={"ID":"5664bd90-a736-4150-b744-fec32ec8dc2f","Type":"ContainerDied","Data":"9bbf42fb9fbe68d2d4234b03c27ad81ed66669e518a331fc830634abc80ee345"} Jul 14 01:46:18.885095 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:18.884754 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7366-140187616088112" event={"ID":"5664bd90-a736-4150-b744-fec32ec8dc2f","Type":"ContainerStarted","Data":"1794ab849af3750b0c5480b3396f52306d3f883eb889a28632cc816f507578ef"} Jul 14 01:46:20.030335 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:20.030300 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7366-140187616088112" Jul 14 01:46:20.049617 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:20.049595 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7366-140187616088112_5664bd90-a736-4150-b744-fec32ec8dc2f/test-curl-7366-140187616088112/0.log" Jul 14 01:46:20.072010 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:20.071979 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7366-140187616088112"] Jul 14 01:46:20.075424 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:20.075389 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7366-140187616088112"] Jul 14 01:46:20.171017 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:20.170925 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqdk8\" (UniqueName: \"kubernetes.io/projected/5664bd90-a736-4150-b744-fec32ec8dc2f-kube-api-access-pqdk8\") pod \"5664bd90-a736-4150-b744-fec32ec8dc2f\" (UID: \"5664bd90-a736-4150-b744-fec32ec8dc2f\") " Jul 14 01:46:20.173129 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:20.173106 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5664bd90-a736-4150-b744-fec32ec8dc2f-kube-api-access-pqdk8" (OuterVolumeSpecName: "kube-api-access-pqdk8") pod "5664bd90-a736-4150-b744-fec32ec8dc2f" (UID: "5664bd90-a736-4150-b744-fec32ec8dc2f"). InnerVolumeSpecName "kube-api-access-pqdk8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:46:20.272470 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:20.272432 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqdk8\" (UniqueName: \"kubernetes.io/projected/5664bd90-a736-4150-b744-fec32ec8dc2f-kube-api-access-pqdk8\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:46:20.897226 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:20.897194 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7366-140187616088112" Jul 14 01:46:20.897404 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:20.897209 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1794ab849af3750b0c5480b3396f52306d3f883eb889a28632cc816f507578ef" Jul 14 01:46:20.907305 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:46:20.907274 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7366-140187616088112\" is forbidden: User \"system:node:ip-10-0-143-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-159.ec2.internal' and this object" podUID="5664bd90-a736-4150-b744-fec32ec8dc2f" pod="openshift-ingress/test-curl-7366-140187616088112" Jul 14 01:46:21.691945 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:46:21.691904 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7366-140187616088112\" is forbidden: User \"system:node:ip-10-0-143-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-159.ec2.internal' and this object" podUID="5664bd90-a736-4150-b744-fec32ec8dc2f" pod="openshift-ingress/test-curl-7366-140187616088112" Jul 14 01:46:21.692719 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:21.692697 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5664bd90-a736-4150-b744-fec32ec8dc2f" path="/var/lib/kubelet/pods/5664bd90-a736-4150-b744-fec32ec8dc2f/volumes" Jul 14 01:46:25.993754 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:25.993710 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7"] Jul 14 01:46:25.994138 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:25.994087 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5664bd90-a736-4150-b744-fec32ec8dc2f" containerName="test-curl-7366-140187616088112" Jul 14 01:46:25.994138 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:25.994098 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5664bd90-a736-4150-b744-fec32ec8dc2f" containerName="test-curl-7366-140187616088112" Jul 14 01:46:25.994214 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:25.994164 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5664bd90-a736-4150-b744-fec32ec8dc2f" containerName="test-curl-7366-140187616088112" Jul 14 01:46:25.999418 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:25.999398 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7" Jul 14 01:46:26.002494 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:26.002463 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 01:46:26.002625 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:26.002515 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zn7l8\"" Jul 14 01:46:26.004054 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:26.004017 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7"] Jul 14 01:46:26.132418 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:26.132385 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18b40d64-d692-4fe7-bee9-be48a799a143-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7\" (UID: \"18b40d64-d692-4fe7-bee9-be48a799a143\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7" Jul 14 01:46:26.233751 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:26.233718 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18b40d64-d692-4fe7-bee9-be48a799a143-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7\" (UID: \"18b40d64-d692-4fe7-bee9-be48a799a143\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7" Jul 14 01:46:26.234372 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:26.234355 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18b40d64-d692-4fe7-bee9-be48a799a143-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7\" (UID: \"18b40d64-d692-4fe7-bee9-be48a799a143\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7" Jul 14 01:46:26.311942 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:26.311836 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7" Jul 14 01:46:26.450721 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:26.450686 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7"] Jul 14 01:46:26.454148 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:46:26.454119 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18b40d64_d692_4fe7_bee9_be48a799a143.slice/crio-e2ae3f64525ef734960662d82e187b9535dcc6c2dc3523b82c79b23d066ccf43 WatchSource:0}: Error finding container e2ae3f64525ef734960662d82e187b9535dcc6c2dc3523b82c79b23d066ccf43: Status 404 returned error can't find the container with id e2ae3f64525ef734960662d82e187b9535dcc6c2dc3523b82c79b23d066ccf43 Jul 14 01:46:26.934881 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:26.934840 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7" event={"ID":"18b40d64-d692-4fe7-bee9-be48a799a143","Type":"ContainerStarted","Data":"8a9a3ac17d425541c2158ab9920359239f30ba75912f5515fe494e10ad3bff49"} Jul 14 01:46:26.935070 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:26.934908 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7" event={"ID":"18b40d64-d692-4fe7-bee9-be48a799a143","Type":"ContainerStarted","Data":"e2ae3f64525ef734960662d82e187b9535dcc6c2dc3523b82c79b23d066ccf43"} Jul 14 01:46:26.951020 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:26.950915 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7" podStartSLOduration=1.950900473 podStartE2EDuration="1.950900473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:46:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:46:26.948952855 +0000 UTC m=+2815.851808356" watchObservedRunningTime="2026-07-14 01:46:26.950900473 +0000 UTC m=+2815.853756019" Jul 14 01:46:27.941783 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:27.941744 2582 generic.go:356] "Generic (PLEG): container finished" podID="18b40d64-d692-4fe7-bee9-be48a799a143" containerID="8a9a3ac17d425541c2158ab9920359239f30ba75912f5515fe494e10ad3bff49" exitCode=0 Jul 14 01:46:27.942202 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:27.941839 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7" event={"ID":"18b40d64-d692-4fe7-bee9-be48a799a143","Type":"ContainerDied","Data":"8a9a3ac17d425541c2158ab9920359239f30ba75912f5515fe494e10ad3bff49"} Jul 14 01:46:29.081408 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:29.081379 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7" Jul 14 01:46:29.162760 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:29.162641 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18b40d64-d692-4fe7-bee9-be48a799a143-maas-api-service-ca\") pod \"18b40d64-d692-4fe7-bee9-be48a799a143\" (UID: \"18b40d64-d692-4fe7-bee9-be48a799a143\") " Jul 14 01:46:29.162992 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:29.162969 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18b40d64-d692-4fe7-bee9-be48a799a143-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "18b40d64-d692-4fe7-bee9-be48a799a143" (UID: "18b40d64-d692-4fe7-bee9-be48a799a143"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:46:29.264040 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:29.263939 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18b40d64-d692-4fe7-bee9-be48a799a143-maas-api-service-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:46:29.953762 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:29.953670 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7" event={"ID":"18b40d64-d692-4fe7-bee9-be48a799a143","Type":"ContainerDied","Data":"e2ae3f64525ef734960662d82e187b9535dcc6c2dc3523b82c79b23d066ccf43"} Jul 14 01:46:29.953762 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:29.953730 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2ae3f64525ef734960662d82e187b9535dcc6c2dc3523b82c79b23d066ccf43" Jul 14 01:46:29.953762 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:29.953680 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7" Jul 14 01:46:36.394701 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:36.394664 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c6543c-29733225-xz9zl"] Jul 14 01:46:36.398650 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:36.398602 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c6543c-29733225-xz9zl"] Jul 14 01:46:37.692617 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:37.692576 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e057386f-526a-4091-81ce-cb3fcdc95f07" path="/var/lib/kubelet/pods/e057386f-526a-4091-81ce-cb3fcdc95f07/volumes" Jul 14 01:46:51.804027 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:51.803989 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s"] Jul 14 01:46:51.804599 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:51.804252 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" podUID="c4231cd7-cf2c-46c1-ac01-a490652861e1" containerName="istio-proxy" containerID="cri-o://b1d46b94257d78a33beec05d9671a8338f40bc5729b7c998c357532a5d739285" gracePeriod=30 Jul 14 01:46:52.191797 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:52.191763 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd"] Jul 14 01:46:52.192128 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:52.192115 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18b40d64-d692-4fe7-bee9-be48a799a143" containerName="revoke-keys" Jul 14 01:46:52.192177 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:52.192130 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="18b40d64-d692-4fe7-bee9-be48a799a143" containerName="revoke-keys" Jul 14 01:46:52.192213 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:52.192191 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="18b40d64-d692-4fe7-bee9-be48a799a143" containerName="revoke-keys" Jul 14 01:46:52.195394 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:52.195367 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd" Jul 14 01:46:52.197889 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:52.197861 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zn7l8\"" Jul 14 01:46:52.198033 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:52.197896 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 01:46:52.202729 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:52.202697 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd"] Jul 14 01:46:52.251049 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:52.251017 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69470985-2de5-4692-96dd-1ebcf24218c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd\" (UID: \"69470985-2de5-4692-96dd-1ebcf24218c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd" Jul 14 01:46:52.351778 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:52.351735 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69470985-2de5-4692-96dd-1ebcf24218c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd\" (UID: \"69470985-2de5-4692-96dd-1ebcf24218c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd" Jul 14 01:46:52.352369 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:52.352351 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69470985-2de5-4692-96dd-1ebcf24218c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd\" (UID: \"69470985-2de5-4692-96dd-1ebcf24218c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd" Jul 14 01:46:52.508956 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:52.508871 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd" Jul 14 01:46:52.652077 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:52.652053 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd"] Jul 14 01:46:52.653833 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:46:52.653751 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69470985_2de5_4692_96dd_1ebcf24218c9.slice/crio-a171a554334bfd4cdbe4b8b151aca921934f85ec2168ce3ee9c33c4a32c27b2b WatchSource:0}: Error finding container a171a554334bfd4cdbe4b8b151aca921934f85ec2168ce3ee9c33c4a32c27b2b: Status 404 returned error can't find the container with id a171a554334bfd4cdbe4b8b151aca921934f85ec2168ce3ee9c33c4a32c27b2b Jul 14 01:46:53.080812 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:53.080770 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd" event={"ID":"69470985-2de5-4692-96dd-1ebcf24218c9","Type":"ContainerStarted","Data":"94d8f74ccd3f73ceb6a9c7449b3ca4d2d372edeb5200e30849cbae8cd177bedd"} Jul 14 01:46:53.081277 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:53.080838 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd" event={"ID":"69470985-2de5-4692-96dd-1ebcf24218c9","Type":"ContainerStarted","Data":"a171a554334bfd4cdbe4b8b151aca921934f85ec2168ce3ee9c33c4a32c27b2b"} Jul 14 01:46:53.096545 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:53.096487 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd" podStartSLOduration=1.096471126 podStartE2EDuration="1.096471126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:46:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:46:53.094253738 +0000 UTC m=+2841.997109231" watchObservedRunningTime="2026-07-14 01:46:53.096471126 +0000 UTC m=+2841.999326625" Jul 14 01:46:54.087349 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:54.087291 2582 generic.go:356] "Generic (PLEG): container finished" podID="69470985-2de5-4692-96dd-1ebcf24218c9" containerID="94d8f74ccd3f73ceb6a9c7449b3ca4d2d372edeb5200e30849cbae8cd177bedd" exitCode=0 Jul 14 01:46:54.087731 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:54.087375 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd" event={"ID":"69470985-2de5-4692-96dd-1ebcf24218c9","Type":"ContainerDied","Data":"94d8f74ccd3f73ceb6a9c7449b3ca4d2d372edeb5200e30849cbae8cd177bedd"} Jul 14 01:46:55.235273 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:55.235248 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd" Jul 14 01:46:55.377651 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:55.377549 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69470985-2de5-4692-96dd-1ebcf24218c9-maas-api-service-ca\") pod \"69470985-2de5-4692-96dd-1ebcf24218c9\" (UID: \"69470985-2de5-4692-96dd-1ebcf24218c9\") " Jul 14 01:46:55.377958 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:55.377930 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69470985-2de5-4692-96dd-1ebcf24218c9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "69470985-2de5-4692-96dd-1ebcf24218c9" (UID: "69470985-2de5-4692-96dd-1ebcf24218c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:46:55.478491 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:55.478446 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69470985-2de5-4692-96dd-1ebcf24218c9-maas-api-service-ca\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:46:56.100392 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:56.100359 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd" Jul 14 01:46:56.100392 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:56.100378 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd" event={"ID":"69470985-2de5-4692-96dd-1ebcf24218c9","Type":"ContainerDied","Data":"a171a554334bfd4cdbe4b8b151aca921934f85ec2168ce3ee9c33c4a32c27b2b"} Jul 14 01:46:56.100623 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:56.100431 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a171a554334bfd4cdbe4b8b151aca921934f85ec2168ce3ee9c33c4a32c27b2b" Jul 14 01:46:57.067555 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.067500 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:46:57.088224 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.088184 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-envoy\") pod \"c4231cd7-cf2c-46c1-ac01-a490652861e1\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " Jul 14 01:46:57.088224 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.088219 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22vdk\" (UniqueName: \"kubernetes.io/projected/c4231cd7-cf2c-46c1-ac01-a490652861e1-kube-api-access-22vdk\") pod \"c4231cd7-cf2c-46c1-ac01-a490652861e1\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " Jul 14 01:46:57.088460 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.088311 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-podinfo\") pod \"c4231cd7-cf2c-46c1-ac01-a490652861e1\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " Jul 14 01:46:57.088460 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.088389 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-data\") pod \"c4231cd7-cf2c-46c1-ac01-a490652861e1\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " Jul 14 01:46:57.088460 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.088425 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-workload-certs\") pod \"c4231cd7-cf2c-46c1-ac01-a490652861e1\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " Jul 14 01:46:57.088460 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.088450 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-token\") pod \"c4231cd7-cf2c-46c1-ac01-a490652861e1\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " Jul 14 01:46:57.088664 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.088556 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-credential-socket\") pod \"c4231cd7-cf2c-46c1-ac01-a490652861e1\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " Jul 14 01:46:57.088664 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.088598 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-workload-socket\") pod \"c4231cd7-cf2c-46c1-ac01-a490652861e1\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " Jul 14 01:46:57.089232 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.089116 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c4231cd7-cf2c-46c1-ac01-a490652861e1" (UID: "c4231cd7-cf2c-46c1-ac01-a490652861e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:46:57.089556 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.089526 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c4231cd7-cf2c-46c1-ac01-a490652861e1" (UID: "c4231cd7-cf2c-46c1-ac01-a490652861e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:46:57.090274 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.090173 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-data" (OuterVolumeSpecName: "istio-data") pod "c4231cd7-cf2c-46c1-ac01-a490652861e1" (UID: "c4231cd7-cf2c-46c1-ac01-a490652861e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:46:57.090274 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.090248 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c4231cd7-cf2c-46c1-ac01-a490652861e1" (UID: "c4231cd7-cf2c-46c1-ac01-a490652861e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:46:57.091279 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.091251 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c4231cd7-cf2c-46c1-ac01-a490652861e1" (UID: "c4231cd7-cf2c-46c1-ac01-a490652861e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:46:57.091591 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.091562 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "c4231cd7-cf2c-46c1-ac01-a490652861e1" (UID: "c4231cd7-cf2c-46c1-ac01-a490652861e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:46:57.091710 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.091645 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4231cd7-cf2c-46c1-ac01-a490652861e1-kube-api-access-22vdk" (OuterVolumeSpecName: "kube-api-access-22vdk") pod "c4231cd7-cf2c-46c1-ac01-a490652861e1" (UID: "c4231cd7-cf2c-46c1-ac01-a490652861e1"). InnerVolumeSpecName "kube-api-access-22vdk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:46:57.092158 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.092132 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c4231cd7-cf2c-46c1-ac01-a490652861e1" (UID: "c4231cd7-cf2c-46c1-ac01-a490652861e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:46:57.107838 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.107805 2582 generic.go:356] "Generic (PLEG): container finished" podID="c4231cd7-cf2c-46c1-ac01-a490652861e1" containerID="b1d46b94257d78a33beec05d9671a8338f40bc5729b7c998c357532a5d739285" exitCode=0 Jul 14 01:46:57.107985 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.107868 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" Jul 14 01:46:57.107985 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.107890 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" event={"ID":"c4231cd7-cf2c-46c1-ac01-a490652861e1","Type":"ContainerDied","Data":"b1d46b94257d78a33beec05d9671a8338f40bc5729b7c998c357532a5d739285"} Jul 14 01:46:57.108093 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.107985 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s" event={"ID":"c4231cd7-cf2c-46c1-ac01-a490652861e1","Type":"ContainerDied","Data":"ec3576118354e9cde72d163ba1aebdae382bf78ab90332cc09096f8132329cd1"} Jul 14 01:46:57.108093 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.108010 2582 scope.go:117] "RemoveContainer" containerID="b1d46b94257d78a33beec05d9671a8338f40bc5729b7c998c357532a5d739285" Jul 14 01:46:57.117578 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.117562 2582 scope.go:117] "RemoveContainer" containerID="de035763c21a7993804c18a482f47119fac517a6949ec9e05f987e416a66f017" Jul 14 01:46:57.125827 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.125811 2582 scope.go:117] "RemoveContainer" containerID="b1d46b94257d78a33beec05d9671a8338f40bc5729b7c998c357532a5d739285" Jul 14 01:46:57.126087 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:46:57.126066 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1d46b94257d78a33beec05d9671a8338f40bc5729b7c998c357532a5d739285\": container with ID starting with b1d46b94257d78a33beec05d9671a8338f40bc5729b7c998c357532a5d739285 not found: ID does not exist" containerID="b1d46b94257d78a33beec05d9671a8338f40bc5729b7c998c357532a5d739285" Jul 14 01:46:57.126154 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.126097 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1d46b94257d78a33beec05d9671a8338f40bc5729b7c998c357532a5d739285"} err="failed to get container status \"b1d46b94257d78a33beec05d9671a8338f40bc5729b7c998c357532a5d739285\": rpc error: code = NotFound desc = could not find container \"b1d46b94257d78a33beec05d9671a8338f40bc5729b7c998c357532a5d739285\": container with ID starting with b1d46b94257d78a33beec05d9671a8338f40bc5729b7c998c357532a5d739285 not found: ID does not exist" Jul 14 01:46:57.126154 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.126114 2582 scope.go:117] "RemoveContainer" containerID="de035763c21a7993804c18a482f47119fac517a6949ec9e05f987e416a66f017" Jul 14 01:46:57.126367 ip-10-0-143-159 kubenswrapper[2582]: E0714 01:46:57.126349 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de035763c21a7993804c18a482f47119fac517a6949ec9e05f987e416a66f017\": container with ID starting with de035763c21a7993804c18a482f47119fac517a6949ec9e05f987e416a66f017 not found: ID does not exist" containerID="de035763c21a7993804c18a482f47119fac517a6949ec9e05f987e416a66f017" Jul 14 01:46:57.126445 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.126374 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de035763c21a7993804c18a482f47119fac517a6949ec9e05f987e416a66f017"} err="failed to get container status \"de035763c21a7993804c18a482f47119fac517a6949ec9e05f987e416a66f017\": rpc error: code = NotFound desc = could not find container \"de035763c21a7993804c18a482f47119fac517a6949ec9e05f987e416a66f017\": container with ID starting with de035763c21a7993804c18a482f47119fac517a6949ec9e05f987e416a66f017 not found: ID does not exist" Jul 14 01:46:57.189691 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.189655 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4231cd7-cf2c-46c1-ac01-a490652861e1-istiod-ca-cert\") pod \"c4231cd7-cf2c-46c1-ac01-a490652861e1\" (UID: \"c4231cd7-cf2c-46c1-ac01-a490652861e1\") " Jul 14 01:46:57.189871 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.189859 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-credential-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:46:57.189919 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.189873 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-workload-socket\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:46:57.189919 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.189883 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-envoy\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:46:57.189919 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.189892 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22vdk\" (UniqueName: \"kubernetes.io/projected/c4231cd7-cf2c-46c1-ac01-a490652861e1-kube-api-access-22vdk\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:46:57.189919 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.189901 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-podinfo\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:46:57.189919 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.189908 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-data\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:46:57.189919 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.189916 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4231cd7-cf2c-46c1-ac01-a490652861e1-workload-certs\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:46:57.190109 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.189923 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4231cd7-cf2c-46c1-ac01-a490652861e1-istio-token\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:46:57.190109 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.189983 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4231cd7-cf2c-46c1-ac01-a490652861e1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c4231cd7-cf2c-46c1-ac01-a490652861e1" (UID: "c4231cd7-cf2c-46c1-ac01-a490652861e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:46:57.290345 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.290286 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4231cd7-cf2c-46c1-ac01-a490652861e1-istiod-ca-cert\") on node \"ip-10-0-143-159.ec2.internal\" DevicePath \"\"" Jul 14 01:46:57.430461 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.430430 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s"] Jul 14 01:46:57.434247 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.434220 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-c6543c-openshift-default-9f484f455-w7d4s"] Jul 14 01:46:57.692703 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:46:57.692613 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4231cd7-cf2c-46c1-ac01-a490652861e1" path="/var/lib/kubelet/pods/c4231cd7-cf2c-46c1-ac01-a490652861e1/volumes" Jul 14 01:47:02.724494 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:02.724456 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a21fee-29733225-lwz4b"] Jul 14 01:47:02.728632 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:02.728603 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a21fee-29733225-lwz4b"] Jul 14 01:47:03.693025 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:03.692989 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18e3359d-9caa-4fd0-9a12-21e2722154c6" path="/var/lib/kubelet/pods/18e3359d-9caa-4fd0-9a12-21e2722154c6/volumes" Jul 14 01:47:24.539678 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:24.539607 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-422x7_eb0445d6-8136-4677-a929-dfd78207f3b4/server/0.log" Jul 14 01:47:24.663930 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:24.663898 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-rp96x_00ba49f8-ce8b-42dc-8396-57c324cbdace/manager/0.log" Jul 14 01:47:24.930456 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:24.930425 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-554b6949c8-8k86v_8d400929-90e2-4141-820a-1770487931c6/copy-manifests/0.log" Jul 14 01:47:24.936489 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:24.936469 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-554b6949c8-8k86v_8d400929-90e2-4141-820a-1770487931c6/manager/0.log" Jul 14 01:47:25.611103 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:25.611075 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29733195-w27m6_886c16a7-5646-463b-91b3-d5a6144aeee5/cleanup/0.log" Jul 14 01:47:25.742847 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:25.742819 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29733210-4cs6x_3513846e-02de-4243-83fc-ca6ae1414edd/cleanup/0.log" Jul 14 01:47:25.865731 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:25.865650 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29733225-4zxpl_bd478d0f-2e83-453f-a61b-be602a5d2416/cleanup/0.log" Jul 14 01:47:25.983897 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:25.983868 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-c626eb13-vxzpd_85fbd4c9-f768-4e18-abc9-186632b6688a/revoke-keys/1.log" Jul 14 01:47:26.107922 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:26.107889 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-e1db3cf4-fh9jg_a07cc313-904d-49ba-9ddd-4cd5167691d2/revoke-keys/0.log" Jul 14 01:47:26.617748 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:26.617714 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-c6eb1c-schjc_621d7a17-8b74-41ac-9501-ca4ac2a2ff01/revoke-keys/0.log" Jul 14 01:47:26.735347 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:26.735298 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-d5004e-2zx2t_c8ff4d73-1690-4638-ab39-33ac9e2cefa1/revoke-keys/0.log" Jul 14 01:47:26.855403 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:26.855372 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-4928854b-rcgqv_414c797b-d977-4668-b1f1-0e7bd3cd06ff/revoke-keys/0.log" Jul 14 01:47:26.973154 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:26.973077 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-40103c45-fr629_325bef2d-f63f-4786-80f8-66e1227b000d/revoke-keys/0.log" Jul 14 01:47:27.096237 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:27.096209 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-45ee56e2-gfrp6_74beae2d-99b9-46ac-afe0-f428e7516634/revoke-keys/0.log" Jul 14 01:47:27.215022 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:27.214992 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-c6543c-vtcn7_18b40d64-d692-4fe7-bee9-be48a799a143/revoke-keys/0.log" Jul 14 01:47:27.331952 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:27.331874 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-a21fee-6t2dd_69470985-2de5-4692-96dd-1ebcf24218c9/revoke-keys/0.log" Jul 14 01:47:29.063288 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:29.063260 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-vzq6c_8cef83ac-b449-4ab8-a93e-a8edff34092f/manager/0.log" Jul 14 01:47:29.588775 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:29.588745 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-xbtm6_c22fd79d-ff1d-4db3-b674-3d51a4fa47ea/manager/0.log" Jul 14 01:47:30.224227 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:30.224198 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4k6q4f_e18dac72-7f63-4046-9994-747715f7d4b2/istio-proxy/0.log" Jul 14 01:47:30.347792 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:30.347759 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ln4jb_9662917b-5681-4a8e-afe3-055f6ebcb23d/discovery/0.log" Jul 14 01:47:30.624146 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:30.624113 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54d956c5b-x5bv2_fdf94a72-0712-48e2-8131-a13bfcd34872/kube-auth-proxy/0.log" Jul 14 01:47:31.774508 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:31.774470 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-9c42l_bb66014b-51f0-48b1-972b-5aa42c9d6fb4/main/0.log" Jul 14 01:47:32.021996 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:32.021962 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-trvdd_1e0d08e0-7561-4713-beff-1962e9415d0c/main/0.log" Jul 14 01:47:39.078653 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:39.078627 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rmnsp_f7953918-10f5-4a00-8b0a-d89348f25314/global-pull-secret-syncer/0.log" Jul 14 01:47:39.178488 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:39.178455 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pgcsp_7d405593-7032-416c-852c-c947b5481d4f/konnectivity-agent/0.log" Jul 14 01:47:39.238580 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:39.238548 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-159.ec2.internal_a7445fbc0edf80cb2d4f53f5ad08f1d5/haproxy/0.log" Jul 14 01:47:43.858522 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:43.858496 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-vzq6c_8cef83ac-b449-4ab8-a93e-a8edff34092f/manager/0.log" Jul 14 01:47:44.062010 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:44.061969 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-xbtm6_c22fd79d-ff1d-4db3-b674-3d51a4fa47ea/manager/0.log" Jul 14 01:47:46.316277 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:46.316249 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ln8nk_99b530e3-52a1-4b82-9c62-f3334a2c8017/node-exporter/0.log" Jul 14 01:47:46.342307 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:46.342279 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ln8nk_99b530e3-52a1-4b82-9c62-f3334a2c8017/kube-rbac-proxy/0.log" Jul 14 01:47:46.369258 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:46.369228 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ln8nk_99b530e3-52a1-4b82-9c62-f3334a2c8017/init-textfile/0.log" Jul 14 01:47:49.165789 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.165758 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/1.log" Jul 14 01:47:49.169740 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.169719 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-wgj6h_47f717ea-28d1-474f-8c21-1742d49d4306/console-operator/2.log" Jul 14 01:47:49.669531 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.669489 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp"] Jul 14 01:47:49.669987 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.669968 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69470985-2de5-4692-96dd-1ebcf24218c9" containerName="revoke-keys" Jul 14 01:47:49.669987 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.669988 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="69470985-2de5-4692-96dd-1ebcf24218c9" containerName="revoke-keys" Jul 14 01:47:49.670120 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.669998 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4231cd7-cf2c-46c1-ac01-a490652861e1" containerName="istio-proxy" Jul 14 01:47:49.670120 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.670005 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4231cd7-cf2c-46c1-ac01-a490652861e1" containerName="istio-proxy" Jul 14 01:47:49.670120 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.670077 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4231cd7-cf2c-46c1-ac01-a490652861e1" containerName="istio-proxy" Jul 14 01:47:49.670120 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.670084 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="69470985-2de5-4692-96dd-1ebcf24218c9" containerName="revoke-keys" Jul 14 01:47:49.670120 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.670093 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4231cd7-cf2c-46c1-ac01-a490652861e1" containerName="istio-proxy" Jul 14 01:47:49.673212 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.673191 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.675586 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.675562 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lrhh6\"/\"default-dockercfg-cbk22\"" Jul 14 01:47:49.675694 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.675588 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lrhh6\"/\"kube-root-ca.crt\"" Jul 14 01:47:49.676817 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.676795 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lrhh6\"/\"openshift-service-ca.crt\"" Jul 14 01:47:49.680261 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.679804 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp"] Jul 14 01:47:49.760933 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.760893 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76d12475-13ab-4d11-bad6-dfa1df513ac6-lib-modules\") pod \"perf-node-gather-daemonset-5x5mp\" (UID: \"76d12475-13ab-4d11-bad6-dfa1df513ac6\") " pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.760933 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.760938 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/76d12475-13ab-4d11-bad6-dfa1df513ac6-proc\") pod \"perf-node-gather-daemonset-5x5mp\" (UID: \"76d12475-13ab-4d11-bad6-dfa1df513ac6\") " pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.761146 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.760982 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4k5f\" (UniqueName: \"kubernetes.io/projected/76d12475-13ab-4d11-bad6-dfa1df513ac6-kube-api-access-j4k5f\") pod \"perf-node-gather-daemonset-5x5mp\" (UID: \"76d12475-13ab-4d11-bad6-dfa1df513ac6\") " pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.761146 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.761061 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/76d12475-13ab-4d11-bad6-dfa1df513ac6-podres\") pod \"perf-node-gather-daemonset-5x5mp\" (UID: \"76d12475-13ab-4d11-bad6-dfa1df513ac6\") " pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.761146 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.761130 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76d12475-13ab-4d11-bad6-dfa1df513ac6-sys\") pod \"perf-node-gather-daemonset-5x5mp\" (UID: \"76d12475-13ab-4d11-bad6-dfa1df513ac6\") " pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.861559 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.861522 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76d12475-13ab-4d11-bad6-dfa1df513ac6-sys\") pod \"perf-node-gather-daemonset-5x5mp\" (UID: \"76d12475-13ab-4d11-bad6-dfa1df513ac6\") " pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.861759 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.861569 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76d12475-13ab-4d11-bad6-dfa1df513ac6-lib-modules\") pod \"perf-node-gather-daemonset-5x5mp\" (UID: \"76d12475-13ab-4d11-bad6-dfa1df513ac6\") " pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.861759 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.861655 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76d12475-13ab-4d11-bad6-dfa1df513ac6-sys\") pod \"perf-node-gather-daemonset-5x5mp\" (UID: \"76d12475-13ab-4d11-bad6-dfa1df513ac6\") " pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.861759 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.861683 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76d12475-13ab-4d11-bad6-dfa1df513ac6-lib-modules\") pod \"perf-node-gather-daemonset-5x5mp\" (UID: \"76d12475-13ab-4d11-bad6-dfa1df513ac6\") " pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.861759 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.861704 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/76d12475-13ab-4d11-bad6-dfa1df513ac6-proc\") pod \"perf-node-gather-daemonset-5x5mp\" (UID: \"76d12475-13ab-4d11-bad6-dfa1df513ac6\") " pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.861759 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.861733 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4k5f\" (UniqueName: \"kubernetes.io/projected/76d12475-13ab-4d11-bad6-dfa1df513ac6-kube-api-access-j4k5f\") pod \"perf-node-gather-daemonset-5x5mp\" (UID: \"76d12475-13ab-4d11-bad6-dfa1df513ac6\") " pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.862015 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.861759 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/76d12475-13ab-4d11-bad6-dfa1df513ac6-proc\") pod \"perf-node-gather-daemonset-5x5mp\" (UID: \"76d12475-13ab-4d11-bad6-dfa1df513ac6\") " pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.862015 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.861785 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/76d12475-13ab-4d11-bad6-dfa1df513ac6-podres\") pod \"perf-node-gather-daemonset-5x5mp\" (UID: \"76d12475-13ab-4d11-bad6-dfa1df513ac6\") " pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.862015 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.861944 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/76d12475-13ab-4d11-bad6-dfa1df513ac6-podres\") pod \"perf-node-gather-daemonset-5x5mp\" (UID: \"76d12475-13ab-4d11-bad6-dfa1df513ac6\") " pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.869509 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.869487 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4k5f\" (UniqueName: \"kubernetes.io/projected/76d12475-13ab-4d11-bad6-dfa1df513ac6-kube-api-access-j4k5f\") pod \"perf-node-gather-daemonset-5x5mp\" (UID: \"76d12475-13ab-4d11-bad6-dfa1df513ac6\") " pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:49.984245 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:49.984158 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:50.118401 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:50.118376 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp"] Jul 14 01:47:50.120034 ip-10-0-143-159 kubenswrapper[2582]: W0714 01:47:50.120003 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod76d12475_13ab_4d11_bad6_dfa1df513ac6.slice/crio-f4b1f9de6e976cb25d48d4340a6f96e9ef081fd5779276369a8d59e28fda2287 WatchSource:0}: Error finding container f4b1f9de6e976cb25d48d4340a6f96e9ef081fd5779276369a8d59e28fda2287: Status 404 returned error can't find the container with id f4b1f9de6e976cb25d48d4340a6f96e9ef081fd5779276369a8d59e28fda2287 Jul 14 01:47:50.328948 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:50.328874 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-7jgwz_791300b3-0b23-48c5-84a8-3b9b52954625/volume-data-source-validator/0.log" Jul 14 01:47:50.400394 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:50.400354 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" event={"ID":"76d12475-13ab-4d11-bad6-dfa1df513ac6","Type":"ContainerStarted","Data":"e3722e6d6c33038531bfc0ea2e9f6f901fbddac2b20fe39942dc4c9e9d5429b3"} Jul 14 01:47:50.400600 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:50.400413 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" event={"ID":"76d12475-13ab-4d11-bad6-dfa1df513ac6","Type":"ContainerStarted","Data":"f4b1f9de6e976cb25d48d4340a6f96e9ef081fd5779276369a8d59e28fda2287"} Jul 14 01:47:50.400600 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:50.400524 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:50.416781 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:50.416730 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" podStartSLOduration=1.416715737 podStartE2EDuration="1.416715737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:47:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:47:50.41517349 +0000 UTC m=+2899.318028991" watchObservedRunningTime="2026-07-14 01:47:50.416715737 +0000 UTC m=+2899.319571236" Jul 14 01:47:51.342055 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:51.342019 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jcbh7_a2342051-9f5f-4acc-bfd9-67d46c0d9389/dns/0.log" Jul 14 01:47:51.362249 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:51.362220 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jcbh7_a2342051-9f5f-4acc-bfd9-67d46c0d9389/kube-rbac-proxy/0.log" Jul 14 01:47:51.478314 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:51.478282 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5hpkv_c31e51b7-7bee-4ce8-9517-a1e6814a2641/dns-node-resolver/0.log" Jul 14 01:47:52.019945 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:52.019912 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5885768549-tk4qh_7d51c30b-b732-4644-8097-b9f3f62271a9/registry/0.log" Jul 14 01:47:52.040707 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:52.040678 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-24x4q_24004bce-6659-4fc1-a3e5-8a8d85103612/node-ca/0.log" Jul 14 01:47:52.953352 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:52.953305 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4k6q4f_e18dac72-7f63-4046-9994-747715f7d4b2/istio-proxy/0.log" Jul 14 01:47:53.066063 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:53.066032 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ln4jb_9662917b-5681-4a8e-afe3-055f6ebcb23d/discovery/0.log" Jul 14 01:47:53.110306 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:53.110277 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54d956c5b-x5bv2_fdf94a72-0712-48e2-8131-a13bfcd34872/kube-auth-proxy/0.log" Jul 14 01:47:53.917504 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:53.917471 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6x4hg_1191145f-7591-45da-9a24-7c20fc700c59/serve-healthcheck-canary/0.log" Jul 14 01:47:54.396292 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:54.396257 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-tvhvj_be11544a-5834-4744-a9d6-a6bd32372ee6/insights-operator/0.log" Jul 14 01:47:54.396807 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:54.396791 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-tvhvj_be11544a-5834-4744-a9d6-a6bd32372ee6/insights-operator/1.log" Jul 14 01:47:54.414548 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:54.414527 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-28m79_f5c67b4e-18dd-4f01-9459-4421a0a4cc64/kube-rbac-proxy/0.log" Jul 14 01:47:54.433736 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:54.433708 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-28m79_f5c67b4e-18dd-4f01-9459-4421a0a4cc64/exporter/0.log" Jul 14 01:47:54.452623 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:54.452599 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-28m79_f5c67b4e-18dd-4f01-9459-4421a0a4cc64/extractor/0.log" Jul 14 01:47:56.415947 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:56.415914 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lrhh6/perf-node-gather-daemonset-5x5mp" Jul 14 01:47:56.817236 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:56.817146 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-422x7_eb0445d6-8136-4677-a929-dfd78207f3b4/server/0.log" Jul 14 01:47:56.883773 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:56.883743 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-rp96x_00ba49f8-ce8b-42dc-8396-57c324cbdace/manager/0.log" Jul 14 01:47:57.030110 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:57.030079 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-554b6949c8-8k86v_8d400929-90e2-4141-820a-1770487931c6/manager/0.log" Jul 14 01:47:57.051113 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:57.051082 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-554b6949c8-8k86v_8d400929-90e2-4141-820a-1770487931c6/copy-manifests/0.log" Jul 14 01:47:58.333855 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:58.333822 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-trs2n_b4dde0a0-4861-48c4-9c1a-17260a17b610/openshift-lws-operator/0.log" Jul 14 01:47:59.545074 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:59.545043 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4k6q4f_e18dac72-7f63-4046-9994-747715f7d4b2/istio-proxy/0.log" Jul 14 01:47:59.650761 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:59.650734 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ln4jb_9662917b-5681-4a8e-afe3-055f6ebcb23d/discovery/0.log" Jul 14 01:47:59.679674 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:47:59.679600 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54d956c5b-x5bv2_fdf94a72-0712-48e2-8131-a13bfcd34872/kube-auth-proxy/0.log" Jul 14 01:48:00.949831 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:00.949802 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4k6q4f_e18dac72-7f63-4046-9994-747715f7d4b2/istio-proxy/0.log" Jul 14 01:48:01.063570 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:01.063536 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ln4jb_9662917b-5681-4a8e-afe3-055f6ebcb23d/discovery/0.log" Jul 14 01:48:01.095624 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:01.095595 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54d956c5b-x5bv2_fdf94a72-0712-48e2-8131-a13bfcd34872/kube-auth-proxy/0.log" Jul 14 01:48:03.851604 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:03.851573 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-74cl4_9c7799c6-a631-4555-b405-cbcf44b861ce/kube-storage-version-migrator-operator/1.log" Jul 14 01:48:03.852612 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:03.852592 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-74cl4_9c7799c6-a631-4555-b405-cbcf44b861ce/kube-storage-version-migrator-operator/0.log" Jul 14 01:48:04.781010 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:04.780978 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7hspr_6a5c2793-deb1-4de9-953f-28ef081786c7/kube-multus-additional-cni-plugins/0.log" Jul 14 01:48:04.799996 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:04.799964 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7hspr_6a5c2793-deb1-4de9-953f-28ef081786c7/egress-router-binary-copy/0.log" Jul 14 01:48:04.817966 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:04.817934 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7hspr_6a5c2793-deb1-4de9-953f-28ef081786c7/cni-plugins/0.log" Jul 14 01:48:04.836460 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:04.836433 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7hspr_6a5c2793-deb1-4de9-953f-28ef081786c7/bond-cni-plugin/0.log" Jul 14 01:48:04.855003 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:04.854976 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7hspr_6a5c2793-deb1-4de9-953f-28ef081786c7/routeoverride-cni/0.log" Jul 14 01:48:04.873058 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:04.873032 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7hspr_6a5c2793-deb1-4de9-953f-28ef081786c7/whereabouts-cni-bincopy/0.log" Jul 14 01:48:04.891024 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:04.890995 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7hspr_6a5c2793-deb1-4de9-953f-28ef081786c7/whereabouts-cni/0.log" Jul 14 01:48:05.218640 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:05.218605 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-hfvn8_7bd7b9b5-3fdf-47de-8717-bb5c6f150fb8/kube-multus/0.log" Jul 14 01:48:05.372106 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:05.372080 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kch6v_3c724b42-2885-4687-970d-9ef08ddf10ff/network-metrics-daemon/0.log" Jul 14 01:48:05.394672 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:05.394644 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kch6v_3c724b42-2885-4687-970d-9ef08ddf10ff/kube-rbac-proxy/0.log" Jul 14 01:48:06.402159 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:06.402127 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pqg6h_3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f/ovn-controller/0.log" Jul 14 01:48:06.417286 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:06.417262 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pqg6h_3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f/ovn-acl-logging/0.log" Jul 14 01:48:06.438172 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:06.438141 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pqg6h_3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f/kube-rbac-proxy-node/0.log" Jul 14 01:48:06.456990 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:06.456961 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pqg6h_3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f/kube-rbac-proxy-ovn-metrics/0.log" Jul 14 01:48:06.472846 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:06.472824 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pqg6h_3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f/northd/0.log" Jul 14 01:48:06.490436 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:06.490405 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pqg6h_3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f/nbdb/0.log" Jul 14 01:48:06.508976 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:06.508949 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pqg6h_3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f/sbdb/0.log" Jul 14 01:48:06.562936 ip-10-0-143-159 kubenswrapper[2582]: I0714 01:48:06.562905 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pqg6h_3b930a04-b1ae-47ac-ab05-d8acf6d4eb7f/ovnkube-controller/0.log"