Jul 24 17:40:04.263237 ip-10-0-137-185 systemd[1]: Starting Kubernetes Kubelet... Jul 24 17:40:04.679761 ip-10-0-137-185 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 17:40:04.679761 ip-10-0-137-185 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 17:40:04.679761 ip-10-0-137-185 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 17:40:04.679761 ip-10-0-137-185 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 17:40:04.679761 ip-10-0-137-185 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 17:40:04.680736 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.680613 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 17:40:04.685001 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.684980 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 17:40:04.685001 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.684996 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 17:40:04.685001 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685002 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 17:40:04.685001 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685006 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 17:40:04.685001 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685010 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685014 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685018 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685024 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685028 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685031 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685035 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685039 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685042 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685046 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685052 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685058 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685063 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685069 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685074 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685078 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685082 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685093 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685097 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 17:40:04.685293 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685102 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685105 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685110 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685113 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685117 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685121 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685126 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685130 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685134 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685138 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685142 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685147 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685151 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685155 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685160 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685165 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685170 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685174 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685179 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685183 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 17:40:04.686027 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685188 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685192 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685196 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685207 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685211 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685215 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685220 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685224 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685229 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685234 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685239 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685244 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685248 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685252 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685256 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685261 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685266 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685271 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685275 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685279 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 17:40:04.686762 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685283 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685287 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685292 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685296 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685300 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685304 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685309 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685313 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685317 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685321 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685326 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685330 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685336 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685340 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685344 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685349 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685353 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685357 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685361 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685366 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 17:40:04.687535 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685370 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685374 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685379 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685383 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685388 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685395 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685400 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685406 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685411 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685416 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685420 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685425 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685429 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685434 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685438 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.685442 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687536 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687548 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687552 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 17:40:04.688051 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687556 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687559 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687562 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687565 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687568 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687571 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687574 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687577 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687579 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687582 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687584 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687587 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687589 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687593 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687597 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687600 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687602 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687605 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687608 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 17:40:04.688517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687610 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687613 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687617 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687622 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687625 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687628 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687630 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687633 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687636 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687638 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687641 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687644 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687647 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687649 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687652 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687655 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687658 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687660 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687663 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687665 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 17:40:04.689035 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687668 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687670 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687673 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687675 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687678 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687681 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687684 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687686 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687690 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687692 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687695 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687697 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687700 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687702 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687706 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687709 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687712 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687715 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687718 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 17:40:04.689539 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687721 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687723 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687726 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687729 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687733 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687736 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687739 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687741 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687744 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687747 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687750 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687753 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687755 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687758 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687760 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687763 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687765 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687768 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687770 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687773 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 17:40:04.690014 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687775 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687778 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687781 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687784 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687787 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687790 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687792 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687795 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687797 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687800 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687802 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687805 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687807 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687810 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687812 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687814 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687818 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.687821 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688216 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688221 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 17:40:04.690500 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688224 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688227 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688231 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688235 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688238 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688241 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688244 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688247 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688249 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688252 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688255 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688257 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688260 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688263 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688266 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688268 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688271 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688274 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688276 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688279 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 17:40:04.691008 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688281 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688284 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688286 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688289 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688291 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688294 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688297 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688304 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688309 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688312 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688314 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688317 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688319 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688322 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688324 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688328 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688330 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688333 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688335 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 17:40:04.691495 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688337 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688340 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688342 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688345 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688347 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688350 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688352 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688355 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688358 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688361 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688363 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688366 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688368 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688371 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688374 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688376 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688379 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688382 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688384 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688387 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 17:40:04.692017 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688390 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688392 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688395 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688397 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688400 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688402 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688405 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688407 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688410 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688412 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688415 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688417 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688420 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688422 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688424 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688427 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688429 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688432 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688434 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688437 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 17:40:04.692503 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688439 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688443 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688446 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688448 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688453 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688457 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688459 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688462 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688466 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688469 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688472 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688475 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688478 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688481 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688484 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688487 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688489 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.688492 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.688559 2561 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.688566 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 17:40:04.692990 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.688974 2561 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.688979 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.688984 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.688987 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.688991 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.688996 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.688999 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689002 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689005 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689009 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689012 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689015 2561 flags.go:64] FLAG: --cgroup-root="" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689019 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689022 2561 flags.go:64] FLAG: --client-ca-file="" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689026 2561 flags.go:64] FLAG: --cloud-provider="external" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689029 2561 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689034 2561 flags.go:64] FLAG: --cluster-domain="" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689037 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689041 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689044 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689047 2561 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689055 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689058 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 17:40:04.693476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689061 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689065 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689068 2561 flags.go:64] FLAG: --contention-profiling="false" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689071 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689074 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689077 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689080 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689084 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689087 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689090 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689093 2561 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689096 2561 flags.go:64] FLAG: --enable-server="true" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689099 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689103 2561 flags.go:64] FLAG: --event-burst="100" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689106 2561 flags.go:64] FLAG: --event-qps="50" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689109 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689112 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689115 2561 flags.go:64] FLAG: --eviction-hard="" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689118 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689121 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689125 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689128 2561 flags.go:64] FLAG: --eviction-soft="" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689132 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689134 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689138 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 17:40:04.694035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689141 2561 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689144 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689147 2561 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689150 2561 flags.go:64] FLAG: --feature-gates="" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689153 2561 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689156 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689159 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689162 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689166 2561 flags.go:64] FLAG: --healthz-port="10248" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689169 2561 flags.go:64] FLAG: --help="false" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689172 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-137-185.ec2.internal" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689175 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689177 2561 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689180 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689184 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689187 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689190 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689193 2561 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689196 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689207 2561 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689210 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689213 2561 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689216 2561 flags.go:64] FLAG: --kube-reserved="" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689219 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 17:40:04.694691 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689222 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689225 2561 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689228 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689231 2561 flags.go:64] FLAG: --lock-file="" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689234 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689237 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689240 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689246 2561 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689249 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689252 2561 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689255 2561 flags.go:64] FLAG: --logging-format="text" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689258 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689261 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689264 2561 flags.go:64] FLAG: --manifest-url="" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689267 2561 flags.go:64] FLAG: --manifest-url-header="" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689271 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689274 2561 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689278 2561 flags.go:64] FLAG: --max-pods="110" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689281 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689284 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689287 2561 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689290 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689293 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689296 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689299 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 17:40:04.695331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689306 2561 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689309 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689312 2561 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689315 2561 flags.go:64] FLAG: --pod-cidr="" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689318 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689323 2561 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689326 2561 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689331 2561 flags.go:64] FLAG: --pods-per-core="0" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689334 2561 flags.go:64] FLAG: --port="10250" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689337 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689340 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-090b245d64ee94b99" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689343 2561 flags.go:64] FLAG: --qos-reserved="" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689346 2561 flags.go:64] FLAG: --read-only-port="10255" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689349 2561 flags.go:64] FLAG: --register-node="true" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689352 2561 flags.go:64] FLAG: --register-with-taints="" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689355 2561 flags.go:64] FLAG: --registry-burst="10" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689359 2561 flags.go:64] FLAG: --registry-qps="5" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689362 2561 flags.go:64] FLAG: --reserved-cpus="" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689365 2561 flags.go:64] FLAG: --reserved-memory="" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689369 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689372 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689374 2561 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689377 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689380 2561 flags.go:64] FLAG: --runonce="false" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689383 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 17:40:04.696014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689386 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689389 2561 flags.go:64] FLAG: --seccomp-default="false" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689392 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689395 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689398 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689401 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689404 2561 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689407 2561 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689410 2561 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689412 2561 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689415 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689418 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689421 2561 flags.go:64] FLAG: --system-cgroups="" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689423 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689431 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689433 2561 flags.go:64] FLAG: --tls-cert-file="" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689436 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689441 2561 flags.go:64] FLAG: --tls-min-version="" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689444 2561 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689447 2561 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689449 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689452 2561 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689455 2561 flags.go:64] FLAG: --v="2" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689459 2561 flags.go:64] FLAG: --version="false" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689466 2561 flags.go:64] FLAG: --vmodule="" Jul 24 17:40:04.696645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689470 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.689473 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690239 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690243 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690247 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690250 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690253 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690255 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690258 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690260 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690263 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690266 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690268 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690271 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690274 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690276 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690279 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690281 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690284 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690286 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 17:40:04.697282 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690289 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690294 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690296 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690299 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690302 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690305 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690307 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690310 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690313 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690315 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690318 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690320 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690323 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690326 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690328 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690331 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690334 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690336 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690339 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690342 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 17:40:04.697769 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690344 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690347 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690350 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690352 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690356 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690359 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690362 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690365 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690368 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690371 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690374 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690377 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690385 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690389 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690392 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690394 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690397 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690401 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690404 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690406 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 17:40:04.698275 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690409 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690411 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690415 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690417 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690420 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690423 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690425 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690429 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690433 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690436 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690439 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690441 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690444 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690447 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690450 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690452 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690455 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690458 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690460 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 17:40:04.698767 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690463 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690465 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690468 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690472 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690475 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690477 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690481 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690484 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690486 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690489 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690493 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690496 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690498 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690501 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690504 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690507 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690509 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690512 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690515 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690517 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 17:40:04.699298 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690520 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 17:40:04.699783 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.690523 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 17:40:04.699783 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.691132 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 17:40:04.699783 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.697970 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 17:40:04.699783 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.697984 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 17:40:04.699783 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.698004 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 17:40:04.699783 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.698010 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 17:40:04.699783 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698070 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 17:40:04.699783 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698074 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 17:40:04.699783 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698078 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 17:40:04.699783 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698080 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 17:40:04.699783 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698083 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 17:40:04.699783 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698086 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 17:40:04.699783 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698090 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 17:40:04.699783 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698094 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698097 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698100 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698103 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698106 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698110 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698113 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698115 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698119 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698121 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698124 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698127 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698129 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698132 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698134 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698137 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698140 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698143 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698146 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698149 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 17:40:04.700148 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698151 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698154 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698156 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698159 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698161 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698164 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698167 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698170 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698174 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698177 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698179 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698182 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698185 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698188 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698190 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698193 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698195 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698198 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698200 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698203 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 17:40:04.700641 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698206 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698208 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698211 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698214 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698217 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698219 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698222 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698224 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698227 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698229 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698232 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698235 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698237 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698240 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698242 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698245 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698247 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698250 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698252 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698255 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 17:40:04.701151 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698257 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698260 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698263 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698265 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698268 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698271 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698275 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698278 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698281 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698284 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698286 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698289 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698292 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698295 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698297 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698300 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698302 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698304 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698307 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 17:40:04.701645 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698310 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 17:40:04.702132 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698312 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 17:40:04.702132 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698314 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 17:40:04.702132 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698317 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 17:40:04.702132 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698320 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 17:40:04.702132 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698322 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 17:40:04.702132 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698325 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 17:40:04.702132 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698328 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 17:40:04.702132 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698331 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 17:40:04.702132 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698333 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 17:40:04.702132 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698336 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 17:40:04.702132 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698338 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 17:40:04.702132 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698341 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 17:40:04.702132 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.698346 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 17:40:04.702132 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698480 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 17:40:04.702132 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698484 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698487 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698490 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698493 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698495 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698498 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698500 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698503 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698506 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698508 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698511 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698513 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698516 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698518 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698521 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698523 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698526 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698528 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698531 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 17:40:04.702506 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698533 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698536 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698538 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698541 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698544 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698546 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698549 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698551 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698554 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698556 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698558 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698561 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698564 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698566 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698569 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698571 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698574 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698576 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698580 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698583 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 17:40:04.703002 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698586 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698589 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698592 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698595 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698597 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698600 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698602 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698605 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698607 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698610 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698612 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698615 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698617 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698620 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698622 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698625 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698628 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698631 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698633 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698635 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 17:40:04.703490 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698638 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698640 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698643 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698645 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698648 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698650 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698652 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698656 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698660 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698663 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698666 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698669 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698671 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698674 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698676 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698679 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698683 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698687 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698689 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 17:40:04.704030 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698692 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698695 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698697 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698700 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698702 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698705 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698707 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698710 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698712 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698715 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698718 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698720 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698723 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698725 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698728 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698730 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698733 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698735 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698738 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 17:40:04.704517 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:04.698740 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 17:40:04.705000 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.698745 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 17:40:04.705000 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.698867 2561 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 17:40:04.705000 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.700664 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 17:40:04.705000 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.701912 2561 server.go:1019] "Starting client certificate rotation" Jul 24 17:40:04.705000 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.702022 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 17:40:04.705000 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.702056 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 17:40:04.705000 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.704273 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 17:40:04.707953 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.707928 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 17:40:04.716819 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.716798 2561 log.go:25] "Validated CRI v1 runtime API" Jul 24 17:40:04.720161 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.720133 2561 log.go:25] "Validated CRI v1 image API" Jul 24 17:40:04.720989 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.720970 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 17:40:04.723421 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.723402 2561 fs.go:135] Filesystem UUIDs: map[3d2c6c47-eb71-4bd8-8176-c9c0ce8234f7:/dev/nvme0n1p3 55e851eb-a15b-4f99-878c-d8734e3f3b21:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 24 17:40:04.723484 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.723422 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 17:40:04.728752 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.728642 2561 manager.go:216] Machine: {Timestamp:2026-07-24 17:40:04.727349288 +0000 UTC m=+0.340355144 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102252 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c170e41f5354ff3875385abcfc01b SystemUUID:ec2c170e-41f5-354f-f387-5385abcfc01b BootID:8a82de14-c209-4eda-b104-cd0bd4e8e5a8 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:45:96:86:33:71 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:45:96:86:33:71 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:4c:91:b3:b3:49 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 17:40:04.728752 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.728747 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 17:40:04.728870 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.728860 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 17:40:04.731785 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.731757 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 17:40:04.731925 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.731789 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-185.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 17:40:04.732543 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.732533 2561 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 17:40:04.732577 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.732545 2561 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 17:40:04.732577 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.732552 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 17:40:04.733236 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.733223 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 17:40:04.733348 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.733332 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 17:40:04.733989 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.733979 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 17:40:04.735597 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.735585 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 24 17:40:04.735727 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.735717 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 17:40:04.735767 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.735749 2561 kubelet.go:486] "Attempting to sync node with API server" Jul 24 17:40:04.735767 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.735766 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 17:40:04.735821 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.735776 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 17:40:04.735821 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.735786 2561 kubelet.go:398] "Adding apiserver pod source" Jul 24 17:40:04.735821 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.735795 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 17:40:04.736181 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.736163 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 17:40:04.736245 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.736185 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 17:40:04.739072 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.739051 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 17:40:04.740794 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.740781 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 17:40:04.740831 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.740816 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 17:40:04.742545 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.742533 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 17:40:04.742579 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.742550 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 17:40:04.742579 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.742556 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 17:40:04.742579 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.742562 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 17:40:04.742579 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.742570 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 17:40:04.742579 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.742576 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 17:40:04.742708 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.742588 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 17:40:04.742708 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.742595 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 17:40:04.742708 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.742602 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 17:40:04.742708 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.742607 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 17:40:04.742708 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.742614 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 17:40:04.742708 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.742620 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 17:40:04.743236 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.743225 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 17:40:04.743274 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.743238 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 17:40:04.746416 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.746402 2561 server.go:1268] "Started kubelet" Jul 24 17:40:04.746502 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.746479 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 17:40:04.746579 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.746528 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 17:40:04.746633 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.746604 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 17:40:04.749369 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.749346 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-185.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 17:40:04.749511 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:04.749484 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-185.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 17:40:04.749511 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:04.749500 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 17:40:04.749789 ip-10-0-137-185 systemd[1]: Started Kubernetes Kubelet. Jul 24 17:40:04.751571 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.751508 2561 server.go:310] "Adding debug handlers to kubelet server" Jul 24 17:40:04.752570 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.752369 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 17:40:04.755173 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:04.755145 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 17:40:04.757957 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:04.756902 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-185.ec2.internal.18c54a8edb3e4b78 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-185.ec2.internal,UID:ip-10-0-137-185.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-185.ec2.internal,},FirstTimestamp:2026-07-24 17:40:04.746374008 +0000 UTC m=+0.359379865,LastTimestamp:2026-07-24 17:40:04.746374008 +0000 UTC m=+0.359379865,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-185.ec2.internal,}" Jul 24 17:40:04.759062 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.759044 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gkr9n" Jul 24 17:40:04.759847 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.759833 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 17:40:04.759847 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.759847 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 17:40:04.760029 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:04.760013 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:04.760440 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.760418 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 17:40:04.760522 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.760432 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 17:40:04.760522 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.760437 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 17:40:04.760626 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.760551 2561 reconstruct.go:97] "Volume reconstruction finished" Jul 24 17:40:04.760626 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.760562 2561 reconciler.go:29] "Reconciler: start to sync state" Jul 24 17:40:04.760959 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.760926 2561 factory.go:153] Registering CRI-O factory Jul 24 17:40:04.761038 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.760963 2561 factory.go:223] Registration of the crio container factory successfully Jul 24 17:40:04.761038 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.761011 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 17:40:04.761038 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.761021 2561 factory.go:55] Registering systemd factory Jul 24 17:40:04.761038 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.761029 2561 factory.go:223] Registration of the systemd container factory successfully Jul 24 17:40:04.761205 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.761050 2561 factory.go:103] Registering Raw factory Jul 24 17:40:04.761205 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.761065 2561 manager.go:1196] Started watching for new ooms in manager Jul 24 17:40:04.761484 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.761471 2561 manager.go:349] Starting recovery of all containers Jul 24 17:40:04.764123 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:04.764096 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 24 17:40:04.764284 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:04.764219 2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-185.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 24 17:40:04.764908 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.764887 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gkr9n" Jul 24 17:40:04.770341 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.770316 2561 manager.go:354] Recovery completed Jul 24 17:40:04.774785 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.774771 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 17:40:04.776916 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.776894 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 24 17:40:04.777021 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.776951 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 17:40:04.777021 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.776969 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 24 17:40:04.777645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.777631 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 17:40:04.777645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.777644 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 17:40:04.777783 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.777652 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 17:40:04.777783 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.777662 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 24 17:40:04.779079 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:04.778988 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-185.ec2.internal.18c54a8edd1057a8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-185.ec2.internal,UID:ip-10-0-137-185.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-185.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-185.ec2.internal,},FirstTimestamp:2026-07-24 17:40:04.776916904 +0000 UTC m=+0.389922765,LastTimestamp:2026-07-24 17:40:04.776916904 +0000 UTC m=+0.389922765,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-185.ec2.internal,}" Jul 24 17:40:04.780325 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.780176 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 24 17:40:04.780412 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.780338 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 17:40:04.780412 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.780353 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 24 17:40:04.781821 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.781804 2561 policy_none.go:49] "None policy: Start" Jul 24 17:40:04.781891 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.781826 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 17:40:04.781891 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.781840 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 17:40:04.782866 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.782851 2561 policy_none.go:47] "Start" Jul 24 17:40:04.825511 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.825495 2561 manager.go:337] "Starting Device Plugin manager" Jul 24 17:40:04.825602 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:04.825534 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 17:40:04.825602 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.825546 2561 server.go:87] "Starting device plugin registration server" Jul 24 17:40:04.825814 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.825801 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 17:40:04.825862 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.825818 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 17:40:04.825914 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.825900 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 17:40:04.825993 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.825982 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 17:40:04.825993 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.825991 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 17:40:04.826587 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:04.826571 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 17:40:04.826668 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:04.826617 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:04.926475 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.926423 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 17:40:04.927745 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.927725 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 24 17:40:04.927859 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.927766 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 17:40:04.927859 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.927781 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 24 17:40:04.927859 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.927817 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-185.ec2.internal" Jul 24 17:40:04.933078 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.933016 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 17:40:04.934249 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.934235 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 17:40:04.934316 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.934274 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 17:40:04.934316 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.934294 2561 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 17:40:04.934411 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:04.934331 2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 17:40:04.936594 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.936573 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 17:40:04.938754 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:04.938738 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-185.ec2.internal" Jul 24 17:40:04.938834 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:04.938759 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-185.ec2.internal\": node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:04.969172 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:04.969143 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:05.035647 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.034911 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal"] Jul 24 17:40:05.035729 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.035692 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 17:40:05.036609 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.036594 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 24 17:40:05.036696 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.036620 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 17:40:05.036696 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.036632 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 24 17:40:05.037762 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.037749 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 17:40:05.037886 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.037872 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 24 17:40:05.037921 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.037902 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 17:40:05.038445 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.038431 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 24 17:40:05.038530 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.038446 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 24 17:40:05.038530 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.038455 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 17:40:05.038530 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.038463 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 17:40:05.038530 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.038466 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 24 17:40:05.038530 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.038475 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 24 17:40:05.039810 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.039797 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 24 17:40:05.039864 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.039820 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 17:40:05.040536 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.040516 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 24 17:40:05.040623 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.040547 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 17:40:05.040623 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.040561 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 24 17:40:05.062750 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.062719 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ac9d4943f96a182d2145019cde9a82c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"3ac9d4943f96a182d2145019cde9a82c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 24 17:40:05.062750 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.062748 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ac9d4943f96a182d2145019cde9a82c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"3ac9d4943f96a182d2145019cde9a82c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 24 17:40:05.062917 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.062769 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/824071a194219c7cb41a1892ae483511-config\") pod \"kube-apiserver-proxy-ip-10-0-137-185.ec2.internal\" (UID: \"824071a194219c7cb41a1892ae483511\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 24 17:40:05.068797 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:05.068777 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-185.ec2.internal\" not found" node="ip-10-0-137-185.ec2.internal" Jul 24 17:40:05.069232 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:05.069216 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:05.073449 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:05.073432 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-185.ec2.internal\" not found" node="ip-10-0-137-185.ec2.internal" Jul 24 17:40:05.163612 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.163579 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/824071a194219c7cb41a1892ae483511-config\") pod \"kube-apiserver-proxy-ip-10-0-137-185.ec2.internal\" (UID: \"824071a194219c7cb41a1892ae483511\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 24 17:40:05.163612 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.163602 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/824071a194219c7cb41a1892ae483511-config\") pod \"kube-apiserver-proxy-ip-10-0-137-185.ec2.internal\" (UID: \"824071a194219c7cb41a1892ae483511\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 24 17:40:05.163826 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.163654 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ac9d4943f96a182d2145019cde9a82c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"3ac9d4943f96a182d2145019cde9a82c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 24 17:40:05.163826 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.163676 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ac9d4943f96a182d2145019cde9a82c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"3ac9d4943f96a182d2145019cde9a82c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 24 17:40:05.163826 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.163718 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ac9d4943f96a182d2145019cde9a82c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"3ac9d4943f96a182d2145019cde9a82c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 24 17:40:05.163826 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.163735 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ac9d4943f96a182d2145019cde9a82c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"3ac9d4943f96a182d2145019cde9a82c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 24 17:40:05.169730 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:05.169710 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:05.270652 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:05.270591 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:05.371085 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:05.371057 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:05.372153 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.372139 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 24 17:40:05.378005 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.377989 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 24 17:40:05.472254 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:05.472223 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:05.572878 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:05.572801 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:05.673552 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:05.673524 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:05.702904 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.702876 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 17:40:05.703441 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.703033 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 17:40:05.728761 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.728738 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 17:40:05.770500 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.770445 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 17:35:05 +0000 UTC" deadline="2028-03-19 00:44:30.645707833 +0000 UTC" Jul 24 17:40:05.770500 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.770495 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14479h4m24.875217089s" Jul 24 17:40:05.774606 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:05.774586 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:05.875031 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:05.874959 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:05.905119 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:05.905076 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod824071a194219c7cb41a1892ae483511.slice/crio-d3e0571a971a8014efad69d2fbccaf3c447f27d48db9d474a2b6a962ebc23b50 WatchSource:0}: Error finding container d3e0571a971a8014efad69d2fbccaf3c447f27d48db9d474a2b6a962ebc23b50: Status 404 returned error can't find the container with id d3e0571a971a8014efad69d2fbccaf3c447f27d48db9d474a2b6a962ebc23b50 Jul 24 17:40:05.905788 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:05.905767 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ac9d4943f96a182d2145019cde9a82c.slice/crio-b84a0e3701927429349c4c1ee50eba6902e5a955396d99c9623311ca31ecb354 WatchSource:0}: Error finding container b84a0e3701927429349c4c1ee50eba6902e5a955396d99c9623311ca31ecb354: Status 404 returned error can't find the container with id b84a0e3701927429349c4c1ee50eba6902e5a955396d99c9623311ca31ecb354 Jul 24 17:40:05.909623 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.909607 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 17:40:05.937558 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.937514 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" event={"ID":"824071a194219c7cb41a1892ae483511","Type":"ContainerStarted","Data":"d3e0571a971a8014efad69d2fbccaf3c447f27d48db9d474a2b6a962ebc23b50"} Jul 24 17:40:05.938568 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:05.938547 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" event={"ID":"3ac9d4943f96a182d2145019cde9a82c","Type":"ContainerStarted","Data":"b84a0e3701927429349c4c1ee50eba6902e5a955396d99c9623311ca31ecb354"} Jul 24 17:40:05.975675 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:05.975648 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:06.052945 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.052907 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 17:40:06.076447 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:06.076419 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:06.176550 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:06.176481 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 24 17:40:06.247698 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.247670 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 17:40:06.260247 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.260223 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 24 17:40:06.271264 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.271239 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 17:40:06.272319 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.272304 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 24 17:40:06.278878 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.278860 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 17:40:06.538943 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.538679 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 17:40:06.736623 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.736586 2561 apiserver.go:52] "Watching apiserver" Jul 24 17:40:06.743315 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.743289 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 17:40:06.743797 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.743770 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv","openshift-dns/node-resolver-rdlc9","openshift-image-registry/node-ca-2r8wb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal","openshift-multus/multus-additional-cni-plugins-fzfj6","openshift-multus/network-metrics-daemon-2fs78","openshift-network-diagnostics/network-check-target-4q8f4","kube-system/konnectivity-agent-5lrd6","kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal","openshift-cluster-node-tuning-operator/tuned-28ktn","openshift-multus/multus-72fk4","openshift-network-operator/iptables-alerter-f9brh","openshift-ovn-kubernetes/ovnkube-node-dwvml","kube-system/global-pull-secret-syncer-zp4x2"] Jul 24 17:40:06.745519 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.745495 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.746796 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.746771 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.748307 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.748284 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2r8wb" Jul 24 17:40:06.748394 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.748321 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 17:40:06.748769 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.748748 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-d8tkk\"" Jul 24 17:40:06.748861 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.748759 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 17:40:06.750891 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.749910 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wb2tl\"" Jul 24 17:40:06.750891 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.750181 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 17:40:06.750891 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.750358 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 17:40:06.750891 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.750511 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 17:40:06.750891 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.750822 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 17:40:06.750891 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.750849 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 17:40:06.750891 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.750855 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 17:40:06.751323 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.751033 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 17:40:06.751323 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.751035 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 17:40:06.751323 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.751145 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 17:40:06.751323 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.751284 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rcrbd\"" Jul 24 17:40:06.751851 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.751821 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:06.755790 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:06.755544 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:06.757780 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.757760 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.758837 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.758802 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:06.759530 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:06.759507 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:06.760593 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.760574 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5lrd6" Jul 24 17:40:06.760844 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.760823 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 17:40:06.760926 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.760855 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 17:40:06.761112 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.761093 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 17:40:06.761189 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.761101 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 17:40:06.761250 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.761209 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 17:40:06.761559 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.761524 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 17:40:06.761842 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.761608 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ttg2z\"" Jul 24 17:40:06.763196 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.763055 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8dt9x\"" Jul 24 17:40:06.763196 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.763165 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 17:40:06.763536 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.763502 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 17:40:06.763692 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.763671 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:06.763834 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:06.763812 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:06.765239 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.765214 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rdlc9" Jul 24 17:40:06.767468 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.766987 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.767468 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.767290 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 17:40:06.767645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.767628 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 17:40:06.768077 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.767629 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-snznd\"" Jul 24 17:40:06.769282 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.769263 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ksktd\"" Jul 24 17:40:06.769518 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.769292 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 17:40:06.769591 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.769356 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 17:40:06.769652 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.769358 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 17:40:06.770283 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.770264 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.772610 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.772591 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f9brh" Jul 24 17:40:06.772798 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.772776 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 17:40:06.772931 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.772912 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-q658w\"" Jul 24 17:40:06.774118 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774088 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-log-socket\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.774212 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774128 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/525c91a9-d541-474e-9c6a-90cd9d3c460d-serviceca\") pod \"node-ca-2r8wb\" (UID: \"525c91a9-d541-474e-9c6a-90cd9d3c460d\") " pod="openshift-image-registry/node-ca-2r8wb" Jul 24 17:40:06.774212 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774159 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhswk\" (UniqueName: \"kubernetes.io/projected/525c91a9-d541-474e-9c6a-90cd9d3c460d-kube-api-access-qhswk\") pod \"node-ca-2r8wb\" (UID: \"525c91a9-d541-474e-9c6a-90cd9d3c460d\") " pod="openshift-image-registry/node-ca-2r8wb" Jul 24 17:40:06.774212 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774188 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-modprobe-d\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.774365 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774214 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-sysctl-conf\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.774365 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774238 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-kubelet\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.774365 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774262 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-run-ovn-kubernetes\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.774365 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774287 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf7nv\" (UniqueName: \"kubernetes.io/projected/7c547f2c-359d-4d0e-b752-5ecf0c57872e-kube-api-access-gf7nv\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.774365 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774313 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/525c91a9-d541-474e-9c6a-90cd9d3c460d-host\") pod \"node-ca-2r8wb\" (UID: \"525c91a9-d541-474e-9c6a-90cd9d3c460d\") " pod="openshift-image-registry/node-ca-2r8wb" Jul 24 17:40:06.774365 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774338 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/397fd802-aa1f-4a3e-bd2b-6835089bbf23-system-cni-dir\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.774636 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774363 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-kubernetes\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.774636 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774400 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-run\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.774636 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774436 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-host\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.774636 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774460 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55qjx\" (UniqueName: \"kubernetes.io/projected/a42c7507-5b58-40f6-ae1e-e42e1b578499-kube-api-access-55qjx\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.774636 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774483 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-etc-openvswitch\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.774636 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774541 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7c547f2c-359d-4d0e-b752-5ecf0c57872e-ovn-node-metrics-cert\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.774636 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774581 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7c547f2c-359d-4d0e-b752-5ecf0c57872e-ovnkube-script-lib\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.775134 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774648 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-run-systemd\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.775134 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774676 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-var-lib-openvswitch\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.775134 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774702 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-cni-bin\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.775134 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774749 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7c547f2c-359d-4d0e-b752-5ecf0c57872e-ovnkube-config\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.775134 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774794 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/397fd802-aa1f-4a3e-bd2b-6835089bbf23-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.775134 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774822 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gktz\" (UniqueName: \"kubernetes.io/projected/397fd802-aa1f-4a3e-bd2b-6835089bbf23-kube-api-access-6gktz\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.775134 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774849 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lgjt\" (UniqueName: \"kubernetes.io/projected/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-kube-api-access-6lgjt\") pod \"network-metrics-daemon-2fs78\" (UID: \"fa9f496c-49f7-4a62-a5a9-1bb903fc553d\") " pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:06.775134 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774870 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 17:40:06.775134 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774900 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25eb2f6d-27ac-4259-bc89-fbcaa6cf085e-konnectivity-ca\") pod \"konnectivity-agent-5lrd6\" (UID: \"25eb2f6d-27ac-4259-bc89-fbcaa6cf085e\") " pod="kube-system/konnectivity-agent-5lrd6" Jul 24 17:40:06.775134 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774937 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-sysconfig\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.775134 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.774962 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-sysctl-d\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.775134 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775047 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-cni-netd\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.775134 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775077 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.775134 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775124 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/397fd802-aa1f-4a3e-bd2b-6835089bbf23-os-release\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775155 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/397fd802-aa1f-4a3e-bd2b-6835089bbf23-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775159 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775192 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25eb2f6d-27ac-4259-bc89-fbcaa6cf085e-agent-certs\") pod \"konnectivity-agent-5lrd6\" (UID: \"25eb2f6d-27ac-4259-bc89-fbcaa6cf085e\") " pod="kube-system/konnectivity-agent-5lrd6" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775210 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qfgz4\"" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775239 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3c435cc-32ae-4849-a167-4cf7d91b6832-kubelet-config\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775266 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-sys\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775292 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775289 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-lib-modules\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775332 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-var-lib-kubelet\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775364 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a42c7507-5b58-40f6-ae1e-e42e1b578499-tmp\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775390 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-slash\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775414 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-run-ovn\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775448 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/397fd802-aa1f-4a3e-bd2b-6835089bbf23-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775494 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775517 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-run-openvswitch\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775555 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-node-log\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775573 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7c547f2c-359d-4d0e-b752-5ecf0c57872e-env-overrides\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.775858 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775593 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs\") pod \"network-metrics-daemon-2fs78\" (UID: \"fa9f496c-49f7-4a62-a5a9-1bb903fc553d\") " pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:06.776762 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775617 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3c435cc-32ae-4849-a167-4cf7d91b6832-dbus\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:06.776762 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775643 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gv8m\" (UniqueName: \"kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m\") pod \"network-check-target-4q8f4\" (UID: \"af9a2eb4-19e8-4a4c-a655-5fd0c7889235\") " pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:06.776762 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775667 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-run-netns\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.776762 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775725 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/397fd802-aa1f-4a3e-bd2b-6835089bbf23-cnibin\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.776762 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775758 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/397fd802-aa1f-4a3e-bd2b-6835089bbf23-cni-binary-copy\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.776762 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775781 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-systemd\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.776762 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775806 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-tuned\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.776762 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.775831 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-systemd-units\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.781293 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.781249 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 17:40:06.801459 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.801437 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qbjr8" Jul 24 17:40:06.807653 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.807633 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qbjr8" Jul 24 17:40:06.861799 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.861768 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 17:40:06.876476 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876447 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-cni-bin\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.876601 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876490 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7c547f2c-359d-4d0e-b752-5ecf0c57872e-ovnkube-config\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.876601 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876518 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gktz\" (UniqueName: \"kubernetes.io/projected/397fd802-aa1f-4a3e-bd2b-6835089bbf23-kube-api-access-6gktz\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.876601 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876559 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-cni-bin\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.876815 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876612 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czz7t\" (UniqueName: \"kubernetes.io/projected/265d5710-707a-4fab-8ddb-b253fc6a3e74-kube-api-access-czz7t\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.876815 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876644 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-cni-netd\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.876815 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876669 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.876815 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876700 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-socket-dir\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.876815 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876726 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-sys-fs\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.876815 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876759 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3c435cc-32ae-4849-a167-4cf7d91b6832-kubelet-config\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:06.876815 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876767 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-cni-netd\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.876815 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876783 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-sys\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.876815 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876782 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876840 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-sys\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876856 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a42c7507-5b58-40f6-ae1e-e42e1b578499-tmp\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876896 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-run-ovn\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876857 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3c435cc-32ae-4849-a167-4cf7d91b6832-kubelet-config\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.876924 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877051 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-run-openvswitch\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877070 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-node-log\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:06.877072 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877097 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-var-lib-kubelet\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877128 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5200e3bb-ae3f-4798-9909-0657616b6c7a-multus-daemon-config\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877129 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-run-openvswitch\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877123 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-run-ovn\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:06.877148 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret podName:a3c435cc-32ae-4849-a167-4cf7d91b6832 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:07.37712188 +0000 UTC m=+2.990127740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret") pod "global-pull-secret-syncer-zp4x2" (UID: "a3c435cc-32ae-4849-a167-4cf7d91b6832") : object "kube-system"/"original-pull-secret" not registered Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877143 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-node-log\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877161 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877182 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7c547f2c-359d-4d0e-b752-5ecf0c57872e-ovnkube-config\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.877382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877208 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e6d525b8-1f5d-4568-ae50-d71eb372ca67-hosts-file\") pod \"node-resolver-rdlc9\" (UID: \"e6d525b8-1f5d-4568-ae50-d71eb372ca67\") " pod="openshift-dns/node-resolver-rdlc9" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877241 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqggz\" (UniqueName: \"kubernetes.io/projected/e6d525b8-1f5d-4568-ae50-d71eb372ca67-kube-api-access-fqggz\") pod \"node-resolver-rdlc9\" (UID: \"e6d525b8-1f5d-4568-ae50-d71eb372ca67\") " pod="openshift-dns/node-resolver-rdlc9" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877278 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3c435cc-32ae-4849-a167-4cf7d91b6832-dbus\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877371 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/397fd802-aa1f-4a3e-bd2b-6835089bbf23-cnibin\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877402 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-multus-socket-dir-parent\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877429 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b6d9717a-5dd0-410d-b016-9f56d775e96e-iptables-alerter-script\") pod \"iptables-alerter-f9brh\" (UID: \"b6d9717a-5dd0-410d-b016-9f56d775e96e\") " pod="openshift-network-operator/iptables-alerter-f9brh" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877456 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-systemd\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877480 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-tuned\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877504 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-systemd-units\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877522 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/397fd802-aa1f-4a3e-bd2b-6835089bbf23-cnibin\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877530 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/525c91a9-d541-474e-9c6a-90cd9d3c460d-serviceca\") pod \"node-ca-2r8wb\" (UID: \"525c91a9-d541-474e-9c6a-90cd9d3c460d\") " pod="openshift-image-registry/node-ca-2r8wb" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877555 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhswk\" (UniqueName: \"kubernetes.io/projected/525c91a9-d541-474e-9c6a-90cd9d3c460d-kube-api-access-qhswk\") pod \"node-ca-2r8wb\" (UID: \"525c91a9-d541-474e-9c6a-90cd9d3c460d\") " pod="openshift-image-registry/node-ca-2r8wb" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877583 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-multus-cni-dir\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877596 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-systemd\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877404 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3c435cc-32ae-4849-a167-4cf7d91b6832-dbus\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877609 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5200e3bb-ae3f-4798-9909-0657616b6c7a-cni-binary-copy\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877636 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-multus-conf-dir\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.878295 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877661 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-modprobe-d\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877685 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-sysctl-conf\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877710 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-run-ovn-kubernetes\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877735 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/525c91a9-d541-474e-9c6a-90cd9d3c460d-host\") pod \"node-ca-2r8wb\" (UID: \"525c91a9-d541-474e-9c6a-90cd9d3c460d\") " pod="openshift-image-registry/node-ca-2r8wb" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877759 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/397fd802-aa1f-4a3e-bd2b-6835089bbf23-system-cni-dir\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877826 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/397fd802-aa1f-4a3e-bd2b-6835089bbf23-os-release\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877854 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25eb2f6d-27ac-4259-bc89-fbcaa6cf085e-agent-certs\") pod \"konnectivity-agent-5lrd6\" (UID: \"25eb2f6d-27ac-4259-bc89-fbcaa6cf085e\") " pod="kube-system/konnectivity-agent-5lrd6" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877886 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5phj\" (UniqueName: \"kubernetes.io/projected/b6d9717a-5dd0-410d-b016-9f56d775e96e-kube-api-access-d5phj\") pod \"iptables-alerter-f9brh\" (UID: \"b6d9717a-5dd0-410d-b016-9f56d775e96e\") " pod="openshift-network-operator/iptables-alerter-f9brh" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877938 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-run\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.877964 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-host\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878007 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55qjx\" (UniqueName: \"kubernetes.io/projected/a42c7507-5b58-40f6-ae1e-e42e1b578499-kube-api-access-55qjx\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878035 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-etc-openvswitch\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878052 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/525c91a9-d541-474e-9c6a-90cd9d3c460d-serviceca\") pod \"node-ca-2r8wb\" (UID: \"525c91a9-d541-474e-9c6a-90cd9d3c460d\") " pod="openshift-image-registry/node-ca-2r8wb" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878062 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7c547f2c-359d-4d0e-b752-5ecf0c57872e-ovnkube-script-lib\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878091 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/397fd802-aa1f-4a3e-bd2b-6835089bbf23-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878121 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lgjt\" (UniqueName: \"kubernetes.io/projected/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-kube-api-access-6lgjt\") pod \"network-metrics-daemon-2fs78\" (UID: \"fa9f496c-49f7-4a62-a5a9-1bb903fc553d\") " pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878146 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-run-systemd\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.879215 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878167 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-systemd-units\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878224 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-var-lib-openvswitch\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878332 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-modprobe-d\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878174 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-var-lib-openvswitch\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878440 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-sysctl-conf\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878464 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25eb2f6d-27ac-4259-bc89-fbcaa6cf085e-konnectivity-ca\") pod \"konnectivity-agent-5lrd6\" (UID: \"25eb2f6d-27ac-4259-bc89-fbcaa6cf085e\") " pod="kube-system/konnectivity-agent-5lrd6" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878491 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/525c91a9-d541-474e-9c6a-90cd9d3c460d-host\") pod \"node-ca-2r8wb\" (UID: \"525c91a9-d541-474e-9c6a-90cd9d3c460d\") " pod="openshift-image-registry/node-ca-2r8wb" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878496 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-system-cni-dir\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878522 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-run-netns\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878531 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/397fd802-aa1f-4a3e-bd2b-6835089bbf23-system-cni-dir\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878547 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trl5n\" (UniqueName: \"kubernetes.io/projected/5200e3bb-ae3f-4798-9909-0657616b6c7a-kube-api-access-trl5n\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878576 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-sysconfig\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878584 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/397fd802-aa1f-4a3e-bd2b-6835089bbf23-os-release\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878667 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-sysctl-d\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878469 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-run-ovn-kubernetes\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878796 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-etc-openvswitch\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878865 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-run\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878922 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-host\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.879763 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.879187 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25eb2f6d-27ac-4259-bc89-fbcaa6cf085e-konnectivity-ca\") pod \"konnectivity-agent-5lrd6\" (UID: \"25eb2f6d-27ac-4259-bc89-fbcaa6cf085e\") " pod="kube-system/konnectivity-agent-5lrd6" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.879258 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-sysconfig\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.879756 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7c547f2c-359d-4d0e-b752-5ecf0c57872e-ovnkube-script-lib\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.879806 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/397fd802-aa1f-4a3e-bd2b-6835089bbf23-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.879820 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-run-systemd\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.878603 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-sysctl-d\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.879859 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/397fd802-aa1f-4a3e-bd2b-6835089bbf23-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.879892 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-os-release\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.879919 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-lib-modules\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.879943 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-var-lib-kubelet\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.879989 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-slash\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880006 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-var-lib-kubelet\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880034 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/397fd802-aa1f-4a3e-bd2b-6835089bbf23-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880045 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/397fd802-aa1f-4a3e-bd2b-6835089bbf23-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880068 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-slash\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880086 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880113 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-var-lib-cni-multus\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.880595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880135 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-lib-modules\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880141 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7c547f2c-359d-4d0e-b752-5ecf0c57872e-env-overrides\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880166 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs\") pod \"network-metrics-daemon-2fs78\" (UID: \"fa9f496c-49f7-4a62-a5a9-1bb903fc553d\") " pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880192 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gv8m\" (UniqueName: \"kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m\") pod \"network-check-target-4q8f4\" (UID: \"af9a2eb4-19e8-4a4c-a655-5fd0c7889235\") " pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880267 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-run-netns\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:06.880276 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880295 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/397fd802-aa1f-4a3e-bd2b-6835089bbf23-cni-binary-copy\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880323 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-hostroot\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:06.880334 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs podName:fa9f496c-49f7-4a62-a5a9-1bb903fc553d nodeName:}" failed. No retries permitted until 2026-07-24 17:40:07.380318515 +0000 UTC m=+2.993324364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs") pod "network-metrics-daemon-2fs78" (UID: "fa9f496c-49f7-4a62-a5a9-1bb903fc553d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880361 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-run-multus-certs\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880388 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6d525b8-1f5d-4568-ae50-d71eb372ca67-tmp-dir\") pod \"node-resolver-rdlc9\" (UID: \"e6d525b8-1f5d-4568-ae50-d71eb372ca67\") " pod="openshift-dns/node-resolver-rdlc9" Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880414 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-log-socket\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880449 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-cnibin\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880477 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-etc-kubernetes\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880504 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-kubelet\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880507 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/397fd802-aa1f-4a3e-bd2b-6835089bbf23-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880530 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf7nv\" (UniqueName: \"kubernetes.io/projected/7c547f2c-359d-4d0e-b752-5ecf0c57872e-kube-api-access-gf7nv\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.881424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880581 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-run-netns\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.882406 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880593 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7c547f2c-359d-4d0e-b752-5ecf0c57872e-env-overrides\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.882406 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880650 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-host-kubelet\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.882406 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880690 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-run-k8s-cni-cncf-io\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.882406 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880727 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-var-lib-cni-bin\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.882406 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880751 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6d9717a-5dd0-410d-b016-9f56d775e96e-host-slash\") pod \"iptables-alerter-f9brh\" (UID: \"b6d9717a-5dd0-410d-b016-9f56d775e96e\") " pod="openshift-network-operator/iptables-alerter-f9brh" Jul 24 17:40:06.882406 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880759 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7c547f2c-359d-4d0e-b752-5ecf0c57872e-log-socket\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.882406 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880787 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-kubernetes\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.882406 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880823 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7c547f2c-359d-4d0e-b752-5ecf0c57872e-ovn-node-metrics-cert\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.882406 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880833 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-kubernetes\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.882406 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880889 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.882406 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880932 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-registration-dir\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.882406 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.880958 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-device-dir\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.882406 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.881033 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/397fd802-aa1f-4a3e-bd2b-6835089bbf23-cni-binary-copy\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.882406 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.881959 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25eb2f6d-27ac-4259-bc89-fbcaa6cf085e-agent-certs\") pod \"konnectivity-agent-5lrd6\" (UID: \"25eb2f6d-27ac-4259-bc89-fbcaa6cf085e\") " pod="kube-system/konnectivity-agent-5lrd6" Jul 24 17:40:06.882406 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.881981 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a42c7507-5b58-40f6-ae1e-e42e1b578499-tmp\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.883224 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.882834 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a42c7507-5b58-40f6-ae1e-e42e1b578499-etc-tuned\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.883507 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.883486 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7c547f2c-359d-4d0e-b752-5ecf0c57872e-ovn-node-metrics-cert\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.885811 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:06.885692 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 17:40:06.885811 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:06.885715 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 17:40:06.885811 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:06.885730 2561 projected.go:196] Error preparing data for projected volume kube-api-access-9gv8m for pod openshift-network-diagnostics/network-check-target-4q8f4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:40:06.885811 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:06.885781 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m podName:af9a2eb4-19e8-4a4c-a655-5fd0c7889235 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:07.385765221 +0000 UTC m=+2.998771071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9gv8m" (UniqueName: "kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m") pod "network-check-target-4q8f4" (UID: "af9a2eb4-19e8-4a4c-a655-5fd0c7889235") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:40:06.888013 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.887743 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gktz\" (UniqueName: \"kubernetes.io/projected/397fd802-aa1f-4a3e-bd2b-6835089bbf23-kube-api-access-6gktz\") pod \"multus-additional-cni-plugins-fzfj6\" (UID: \"397fd802-aa1f-4a3e-bd2b-6835089bbf23\") " pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:06.888232 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.888206 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lgjt\" (UniqueName: \"kubernetes.io/projected/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-kube-api-access-6lgjt\") pod \"network-metrics-daemon-2fs78\" (UID: \"fa9f496c-49f7-4a62-a5a9-1bb903fc553d\") " pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:06.890055 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.890005 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhswk\" (UniqueName: \"kubernetes.io/projected/525c91a9-d541-474e-9c6a-90cd9d3c460d-kube-api-access-qhswk\") pod \"node-ca-2r8wb\" (UID: \"525c91a9-d541-474e-9c6a-90cd9d3c460d\") " pod="openshift-image-registry/node-ca-2r8wb" Jul 24 17:40:06.891129 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.891084 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf7nv\" (UniqueName: \"kubernetes.io/projected/7c547f2c-359d-4d0e-b752-5ecf0c57872e-kube-api-access-gf7nv\") pod \"ovnkube-node-dwvml\" (UID: \"7c547f2c-359d-4d0e-b752-5ecf0c57872e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:06.891268 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.891223 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55qjx\" (UniqueName: \"kubernetes.io/projected/a42c7507-5b58-40f6-ae1e-e42e1b578499-kube-api-access-55qjx\") pod \"tuned-28ktn\" (UID: \"a42c7507-5b58-40f6-ae1e-e42e1b578499\") " pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:06.981746 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.981716 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-var-lib-kubelet\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.981746 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.981748 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5200e3bb-ae3f-4798-9909-0657616b6c7a-multus-daemon-config\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.981768 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e6d525b8-1f5d-4568-ae50-d71eb372ca67-hosts-file\") pod \"node-resolver-rdlc9\" (UID: \"e6d525b8-1f5d-4568-ae50-d71eb372ca67\") " pod="openshift-dns/node-resolver-rdlc9" Jul 24 17:40:06.982014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.981828 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqggz\" (UniqueName: \"kubernetes.io/projected/e6d525b8-1f5d-4568-ae50-d71eb372ca67-kube-api-access-fqggz\") pod \"node-resolver-rdlc9\" (UID: \"e6d525b8-1f5d-4568-ae50-d71eb372ca67\") " pod="openshift-dns/node-resolver-rdlc9" Jul 24 17:40:06.982014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.981874 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-multus-socket-dir-parent\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.981880 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e6d525b8-1f5d-4568-ae50-d71eb372ca67-hosts-file\") pod \"node-resolver-rdlc9\" (UID: \"e6d525b8-1f5d-4568-ae50-d71eb372ca67\") " pod="openshift-dns/node-resolver-rdlc9" Jul 24 17:40:06.982014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.981902 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b6d9717a-5dd0-410d-b016-9f56d775e96e-iptables-alerter-script\") pod \"iptables-alerter-f9brh\" (UID: \"b6d9717a-5dd0-410d-b016-9f56d775e96e\") " pod="openshift-network-operator/iptables-alerter-f9brh" Jul 24 17:40:06.982014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.981933 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-multus-cni-dir\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.981980 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5200e3bb-ae3f-4798-9909-0657616b6c7a-cni-binary-copy\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982005 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-multus-conf-dir\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982010 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-multus-socket-dir-parent\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982076 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5phj\" (UniqueName: \"kubernetes.io/projected/b6d9717a-5dd0-410d-b016-9f56d775e96e-kube-api-access-d5phj\") pod \"iptables-alerter-f9brh\" (UID: \"b6d9717a-5dd0-410d-b016-9f56d775e96e\") " pod="openshift-network-operator/iptables-alerter-f9brh" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982193 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-multus-cni-dir\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982123 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-system-cni-dir\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982237 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-multus-conf-dir\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982240 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-run-netns\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982274 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-run-netns\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982282 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trl5n\" (UniqueName: \"kubernetes.io/projected/5200e3bb-ae3f-4798-9909-0657616b6c7a-kube-api-access-trl5n\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982288 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-var-lib-kubelet\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982313 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-os-release\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982333 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-system-cni-dir\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982346 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982372 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-var-lib-cni-multus\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982424 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-os-release\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982426 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-hostroot\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982463 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-hostroot\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982464 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-run-multus-certs\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.982483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982495 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-run-multus-certs\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982496 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6d525b8-1f5d-4568-ae50-d71eb372ca67-tmp-dir\") pod \"node-resolver-rdlc9\" (UID: \"e6d525b8-1f5d-4568-ae50-d71eb372ca67\") " pod="openshift-dns/node-resolver-rdlc9" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982517 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982532 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-cnibin\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982538 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b6d9717a-5dd0-410d-b016-9f56d775e96e-iptables-alerter-script\") pod \"iptables-alerter-f9brh\" (UID: \"b6d9717a-5dd0-410d-b016-9f56d775e96e\") " pod="openshift-network-operator/iptables-alerter-f9brh" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982591 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-var-lib-cni-multus\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982593 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-etc-kubernetes\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982616 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-cnibin\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982637 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-etc-kubernetes\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982652 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-run-k8s-cni-cncf-io\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982686 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-var-lib-cni-bin\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982712 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6d9717a-5dd0-410d-b016-9f56d775e96e-host-slash\") pod \"iptables-alerter-f9brh\" (UID: \"b6d9717a-5dd0-410d-b016-9f56d775e96e\") " pod="openshift-network-operator/iptables-alerter-f9brh" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982722 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-run-k8s-cni-cncf-io\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982740 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982752 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5200e3bb-ae3f-4798-9909-0657616b6c7a-host-var-lib-cni-bin\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982772 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-registration-dir\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982778 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6d9717a-5dd0-410d-b016-9f56d775e96e-host-slash\") pod \"iptables-alerter-f9brh\" (UID: \"b6d9717a-5dd0-410d-b016-9f56d775e96e\") " pod="openshift-network-operator/iptables-alerter-f9brh" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982774 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6d525b8-1f5d-4568-ae50-d71eb372ca67-tmp-dir\") pod \"node-resolver-rdlc9\" (UID: \"e6d525b8-1f5d-4568-ae50-d71eb372ca67\") " pod="openshift-dns/node-resolver-rdlc9" Jul 24 17:40:06.983409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982813 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-device-dir\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.984207 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982830 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-registration-dir\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.984207 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982839 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.984207 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982848 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czz7t\" (UniqueName: \"kubernetes.io/projected/265d5710-707a-4fab-8ddb-b253fc6a3e74-kube-api-access-czz7t\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.984207 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982879 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-socket-dir\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.984207 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982885 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-device-dir\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.984207 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.982905 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-sys-fs\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.984207 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.983025 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-sys-fs\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.984207 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.983025 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/265d5710-707a-4fab-8ddb-b253fc6a3e74-socket-dir\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.984207 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.983248 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5200e3bb-ae3f-4798-9909-0657616b6c7a-multus-daemon-config\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.984207 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.983699 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5200e3bb-ae3f-4798-9909-0657616b6c7a-cni-binary-copy\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.994134 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.994103 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqggz\" (UniqueName: \"kubernetes.io/projected/e6d525b8-1f5d-4568-ae50-d71eb372ca67-kube-api-access-fqggz\") pod \"node-resolver-rdlc9\" (UID: \"e6d525b8-1f5d-4568-ae50-d71eb372ca67\") " pod="openshift-dns/node-resolver-rdlc9" Jul 24 17:40:06.994541 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.994515 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czz7t\" (UniqueName: \"kubernetes.io/projected/265d5710-707a-4fab-8ddb-b253fc6a3e74-kube-api-access-czz7t\") pod \"aws-ebs-csi-driver-node-p4cdv\" (UID: \"265d5710-707a-4fab-8ddb-b253fc6a3e74\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:06.995646 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.995597 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trl5n\" (UniqueName: \"kubernetes.io/projected/5200e3bb-ae3f-4798-9909-0657616b6c7a-kube-api-access-trl5n\") pod \"multus-72fk4\" (UID: \"5200e3bb-ae3f-4798-9909-0657616b6c7a\") " pod="openshift-multus/multus-72fk4" Jul 24 17:40:06.996292 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:06.996257 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5phj\" (UniqueName: \"kubernetes.io/projected/b6d9717a-5dd0-410d-b016-9f56d775e96e-kube-api-access-d5phj\") pod \"iptables-alerter-f9brh\" (UID: \"b6d9717a-5dd0-410d-b016-9f56d775e96e\") " pod="openshift-network-operator/iptables-alerter-f9brh" Jul 24 17:40:07.065341 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:07.065259 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-28ktn" Jul 24 17:40:07.073526 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:07.073501 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:07.083496 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:07.083472 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2r8wb" Jul 24 17:40:07.090209 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:07.090183 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fzfj6" Jul 24 17:40:07.099131 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:07.099100 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5lrd6" Jul 24 17:40:07.107401 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:07.107379 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rdlc9" Jul 24 17:40:07.116149 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:07.116128 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" Jul 24 17:40:07.124699 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:07.124680 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-72fk4" Jul 24 17:40:07.135294 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:07.135275 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f9brh" Jul 24 17:40:07.385635 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:07.385545 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:07.385635 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:07.385614 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs\") pod \"network-metrics-daemon-2fs78\" (UID: \"fa9f496c-49f7-4a62-a5a9-1bb903fc553d\") " pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:07.385862 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:07.385749 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:40:07.385862 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:07.385823 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs podName:fa9f496c-49f7-4a62-a5a9-1bb903fc553d nodeName:}" failed. No retries permitted until 2026-07-24 17:40:08.385801626 +0000 UTC m=+3.998807487 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs") pod "network-metrics-daemon-2fs78" (UID: "fa9f496c-49f7-4a62-a5a9-1bb903fc553d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:40:07.385862 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:07.385752 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 17:40:07.386019 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:07.385875 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret podName:a3c435cc-32ae-4849-a167-4cf7d91b6832 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:08.3858643 +0000 UTC m=+3.998870147 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret") pod "global-pull-secret-syncer-zp4x2" (UID: "a3c435cc-32ae-4849-a167-4cf7d91b6832") : object "kube-system"/"original-pull-secret" not registered Jul 24 17:40:07.486366 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:07.486320 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gv8m\" (UniqueName: \"kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m\") pod \"network-check-target-4q8f4\" (UID: \"af9a2eb4-19e8-4a4c-a655-5fd0c7889235\") " pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:07.486557 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:07.486503 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 17:40:07.486557 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:07.486530 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 17:40:07.486557 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:07.486543 2561 projected.go:196] Error preparing data for projected volume kube-api-access-9gv8m for pod openshift-network-diagnostics/network-check-target-4q8f4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:40:07.486711 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:07.486615 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m podName:af9a2eb4-19e8-4a4c-a655-5fd0c7889235 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:08.486594316 +0000 UTC m=+4.099600187 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gv8m" (UniqueName: "kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m") pod "network-check-target-4q8f4" (UID: "af9a2eb4-19e8-4a4c-a655-5fd0c7889235") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:40:07.810672 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:07.810567 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 17:35:07 +0000 UTC" deadline="2027-12-26 07:30:18.20902975 +0000 UTC" Jul 24 17:40:07.810672 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:07.810616 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12469h50m10.398417454s" Jul 24 17:40:07.935138 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:07.935094 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:07.935336 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:07.935238 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:08.021120 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:08.021092 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod397fd802_aa1f_4a3e_bd2b_6835089bbf23.slice/crio-064c72e825693fa1db27d4d5f8b474e75c014ce0355c6eb296fd6bbd6734a406 WatchSource:0}: Error finding container 064c72e825693fa1db27d4d5f8b474e75c014ce0355c6eb296fd6bbd6734a406: Status 404 returned error can't find the container with id 064c72e825693fa1db27d4d5f8b474e75c014ce0355c6eb296fd6bbd6734a406 Jul 24 17:40:08.022759 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:08.022684 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c547f2c_359d_4d0e_b752_5ecf0c57872e.slice/crio-2ca4d2347260ad815de8ab8fdff1eeba7e4674f1e1843eb886d0e8f332a7ba30 WatchSource:0}: Error finding container 2ca4d2347260ad815de8ab8fdff1eeba7e4674f1e1843eb886d0e8f332a7ba30: Status 404 returned error can't find the container with id 2ca4d2347260ad815de8ab8fdff1eeba7e4674f1e1843eb886d0e8f332a7ba30 Jul 24 17:40:08.023624 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:08.023463 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d9717a_5dd0_410d_b016_9f56d775e96e.slice/crio-362cca8fa57eddd44c3be29eced0da4bf5bfe0ff677574429468dd4b5164ecb2 WatchSource:0}: Error finding container 362cca8fa57eddd44c3be29eced0da4bf5bfe0ff677574429468dd4b5164ecb2: Status 404 returned error can't find the container with id 362cca8fa57eddd44c3be29eced0da4bf5bfe0ff677574429468dd4b5164ecb2 Jul 24 17:40:08.027972 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:08.027659 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d525b8_1f5d_4568_ae50_d71eb372ca67.slice/crio-f9be3976cfc8d4373ba1cb992d9f424590a924ca3d03efbc9456bbc78aabf2a1 WatchSource:0}: Error finding container f9be3976cfc8d4373ba1cb992d9f424590a924ca3d03efbc9456bbc78aabf2a1: Status 404 returned error can't find the container with id f9be3976cfc8d4373ba1cb992d9f424590a924ca3d03efbc9456bbc78aabf2a1 Jul 24 17:40:08.031917 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:08.031563 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25eb2f6d_27ac_4259_bc89_fbcaa6cf085e.slice/crio-beec0227bbb897f7bc1c933a3d21353a28d25115593c4c90c3e1d3d9cc5e17f7 WatchSource:0}: Error finding container beec0227bbb897f7bc1c933a3d21353a28d25115593c4c90c3e1d3d9cc5e17f7: Status 404 returned error can't find the container with id beec0227bbb897f7bc1c933a3d21353a28d25115593c4c90c3e1d3d9cc5e17f7 Jul 24 17:40:08.033942 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:08.033608 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda42c7507_5b58_40f6_ae1e_e42e1b578499.slice/crio-cdc184cbb00bc2df199b27ec7f5d0573a89d9b48dfa0d8428ec996516bd313a2 WatchSource:0}: Error finding container cdc184cbb00bc2df199b27ec7f5d0573a89d9b48dfa0d8428ec996516bd313a2: Status 404 returned error can't find the container with id cdc184cbb00bc2df199b27ec7f5d0573a89d9b48dfa0d8428ec996516bd313a2 Jul 24 17:40:08.033942 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:08.033923 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5200e3bb_ae3f_4798_9909_0657616b6c7a.slice/crio-d22f919f5d9dd0452244df6a8f713a7c60e39117881e13c0d9f9729e5699723d WatchSource:0}: Error finding container d22f919f5d9dd0452244df6a8f713a7c60e39117881e13c0d9f9729e5699723d: Status 404 returned error can't find the container with id d22f919f5d9dd0452244df6a8f713a7c60e39117881e13c0d9f9729e5699723d Jul 24 17:40:08.394534 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:08.394242 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs\") pod \"network-metrics-daemon-2fs78\" (UID: \"fa9f496c-49f7-4a62-a5a9-1bb903fc553d\") " pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:08.394534 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:08.394509 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:08.394534 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:08.394414 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:40:08.394792 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:08.394594 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs podName:fa9f496c-49f7-4a62-a5a9-1bb903fc553d nodeName:}" failed. No retries permitted until 2026-07-24 17:40:10.394573095 +0000 UTC m=+6.007578941 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs") pod "network-metrics-daemon-2fs78" (UID: "fa9f496c-49f7-4a62-a5a9-1bb903fc553d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:40:08.394792 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:08.394629 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 17:40:08.394792 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:08.394674 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret podName:a3c435cc-32ae-4849-a167-4cf7d91b6832 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:10.394662646 +0000 UTC m=+6.007668492 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret") pod "global-pull-secret-syncer-zp4x2" (UID: "a3c435cc-32ae-4849-a167-4cf7d91b6832") : object "kube-system"/"original-pull-secret" not registered Jul 24 17:40:08.496064 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:08.496030 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gv8m\" (UniqueName: \"kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m\") pod \"network-check-target-4q8f4\" (UID: \"af9a2eb4-19e8-4a4c-a655-5fd0c7889235\") " pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:08.496233 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:08.496215 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 17:40:08.496273 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:08.496239 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 17:40:08.496273 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:08.496250 2561 projected.go:196] Error preparing data for projected volume kube-api-access-9gv8m for pod openshift-network-diagnostics/network-check-target-4q8f4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:40:08.496334 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:08.496302 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m podName:af9a2eb4-19e8-4a4c-a655-5fd0c7889235 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:10.496287459 +0000 UTC m=+6.109293308 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gv8m" (UniqueName: "kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m") pod "network-check-target-4q8f4" (UID: "af9a2eb4-19e8-4a4c-a655-5fd0c7889235") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:40:08.811802 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:08.811701 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 17:35:07 +0000 UTC" deadline="2028-03-17 17:58:07.148964254 +0000 UTC" Jul 24 17:40:08.811802 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:08.811747 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14448h17m58.337221837s" Jul 24 17:40:08.938268 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:08.937524 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:08.938268 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:08.937664 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:08.938268 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:08.938077 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:08.938268 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:08.938193 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:08.964977 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:08.964922 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rdlc9" event={"ID":"e6d525b8-1f5d-4568-ae50-d71eb372ca67","Type":"ContainerStarted","Data":"f9be3976cfc8d4373ba1cb992d9f424590a924ca3d03efbc9456bbc78aabf2a1"} Jul 24 17:40:08.970208 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:08.970109 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" event={"ID":"7c547f2c-359d-4d0e-b752-5ecf0c57872e","Type":"ContainerStarted","Data":"2ca4d2347260ad815de8ab8fdff1eeba7e4674f1e1843eb886d0e8f332a7ba30"} Jul 24 17:40:08.981563 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:08.980865 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" event={"ID":"824071a194219c7cb41a1892ae483511","Type":"ContainerStarted","Data":"829d0bdd6e87519c6bb1febadee63304acbd7589cc27134ac098efdc71fb01f5"} Jul 24 17:40:08.984155 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:08.984060 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-72fk4" event={"ID":"5200e3bb-ae3f-4798-9909-0657616b6c7a","Type":"ContainerStarted","Data":"d22f919f5d9dd0452244df6a8f713a7c60e39117881e13c0d9f9729e5699723d"} Jul 24 17:40:08.989274 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:08.989192 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2r8wb" event={"ID":"525c91a9-d541-474e-9c6a-90cd9d3c460d","Type":"ContainerStarted","Data":"78deb928d6a428b707a680fb9b308e3c9310d61f6ed7c4440b4e14b07868c0ff"} Jul 24 17:40:09.007695 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:09.007647 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" event={"ID":"265d5710-707a-4fab-8ddb-b253fc6a3e74","Type":"ContainerStarted","Data":"8b05b63e99f30626fd4d824db94dc436c58d9903e635af1cffdd14d9e276efd5"} Jul 24 17:40:09.024653 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:09.024613 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f9brh" event={"ID":"b6d9717a-5dd0-410d-b016-9f56d775e96e","Type":"ContainerStarted","Data":"362cca8fa57eddd44c3be29eced0da4bf5bfe0ff677574429468dd4b5164ecb2"} Jul 24 17:40:09.032209 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:09.032130 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzfj6" event={"ID":"397fd802-aa1f-4a3e-bd2b-6835089bbf23","Type":"ContainerStarted","Data":"064c72e825693fa1db27d4d5f8b474e75c014ce0355c6eb296fd6bbd6734a406"} Jul 24 17:40:09.038485 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:09.038321 2561 generic.go:356] "Generic (PLEG): container finished" podID="3ac9d4943f96a182d2145019cde9a82c" containerID="da2bdca6db91fb120bb8f7e3def93e6023ca7de6346786d8ac40862a4a5ad5f0" exitCode=0 Jul 24 17:40:09.038485 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:09.038416 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" event={"ID":"3ac9d4943f96a182d2145019cde9a82c","Type":"ContainerDied","Data":"da2bdca6db91fb120bb8f7e3def93e6023ca7de6346786d8ac40862a4a5ad5f0"} Jul 24 17:40:09.045432 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:09.045409 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-28ktn" event={"ID":"a42c7507-5b58-40f6-ae1e-e42e1b578499","Type":"ContainerStarted","Data":"cdc184cbb00bc2df199b27ec7f5d0573a89d9b48dfa0d8428ec996516bd313a2"} Jul 24 17:40:09.049808 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:09.049777 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5lrd6" event={"ID":"25eb2f6d-27ac-4259-bc89-fbcaa6cf085e","Type":"ContainerStarted","Data":"beec0227bbb897f7bc1c933a3d21353a28d25115593c4c90c3e1d3d9cc5e17f7"} Jul 24 17:40:09.056523 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:09.056469 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" podStartSLOduration=3.056449686 podStartE2EDuration="3.056449686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:40:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:09.006702148 +0000 UTC m=+4.619708018" watchObservedRunningTime="2026-07-24 17:40:09.056449686 +0000 UTC m=+4.669455559" Jul 24 17:40:09.936112 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:09.935510 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:09.936112 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:09.935653 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:10.068329 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:10.068288 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" event={"ID":"3ac9d4943f96a182d2145019cde9a82c","Type":"ContainerStarted","Data":"71f503e68b203809202c2ff14be3c54e0589843284e7248b4d7bd79f18e6a3cf"} Jul 24 17:40:10.419415 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:10.419308 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:10.419581 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:10.419434 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs\") pod \"network-metrics-daemon-2fs78\" (UID: \"fa9f496c-49f7-4a62-a5a9-1bb903fc553d\") " pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:10.419642 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:10.419618 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:40:10.419704 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:10.419684 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs podName:fa9f496c-49f7-4a62-a5a9-1bb903fc553d nodeName:}" failed. No retries permitted until 2026-07-24 17:40:14.419663145 +0000 UTC m=+10.032668996 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs") pod "network-metrics-daemon-2fs78" (UID: "fa9f496c-49f7-4a62-a5a9-1bb903fc553d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:40:10.420155 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:10.420132 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 17:40:10.420246 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:10.420197 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret podName:a3c435cc-32ae-4849-a167-4cf7d91b6832 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:14.420178245 +0000 UTC m=+10.033184094 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret") pod "global-pull-secret-syncer-zp4x2" (UID: "a3c435cc-32ae-4849-a167-4cf7d91b6832") : object "kube-system"/"original-pull-secret" not registered Jul 24 17:40:10.519886 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:10.519840 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gv8m\" (UniqueName: \"kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m\") pod \"network-check-target-4q8f4\" (UID: \"af9a2eb4-19e8-4a4c-a655-5fd0c7889235\") " pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:10.520108 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:10.520089 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 17:40:10.520175 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:10.520121 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 17:40:10.520175 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:10.520135 2561 projected.go:196] Error preparing data for projected volume kube-api-access-9gv8m for pod openshift-network-diagnostics/network-check-target-4q8f4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:40:10.520291 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:10.520197 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m podName:af9a2eb4-19e8-4a4c-a655-5fd0c7889235 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:14.520177022 +0000 UTC m=+10.133182873 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gv8m" (UniqueName: "kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m") pod "network-check-target-4q8f4" (UID: "af9a2eb4-19e8-4a4c-a655-5fd0c7889235") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:40:10.937340 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:10.937308 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:10.937850 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:10.937459 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:10.937850 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:10.937540 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:10.937850 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:10.937619 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:11.935441 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:11.935407 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:11.935628 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:11.935542 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:12.935374 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:12.935170 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:12.935374 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:12.935300 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:12.935374 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:12.935170 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:12.935972 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:12.935524 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:13.935118 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:13.935035 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:13.935269 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:13.935190 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:14.452729 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:14.452141 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:14.452729 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:14.452218 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs\") pod \"network-metrics-daemon-2fs78\" (UID: \"fa9f496c-49f7-4a62-a5a9-1bb903fc553d\") " pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:14.452729 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:14.452349 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:40:14.452729 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:14.452403 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs podName:fa9f496c-49f7-4a62-a5a9-1bb903fc553d nodeName:}" failed. No retries permitted until 2026-07-24 17:40:22.452385057 +0000 UTC m=+18.065390904 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs") pod "network-metrics-daemon-2fs78" (UID: "fa9f496c-49f7-4a62-a5a9-1bb903fc553d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:40:14.452729 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:14.452478 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 17:40:14.452729 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:14.452542 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret podName:a3c435cc-32ae-4849-a167-4cf7d91b6832 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:22.452528016 +0000 UTC m=+18.065533869 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret") pod "global-pull-secret-syncer-zp4x2" (UID: "a3c435cc-32ae-4849-a167-4cf7d91b6832") : object "kube-system"/"original-pull-secret" not registered Jul 24 17:40:14.553316 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:14.553232 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gv8m\" (UniqueName: \"kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m\") pod \"network-check-target-4q8f4\" (UID: \"af9a2eb4-19e8-4a4c-a655-5fd0c7889235\") " pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:14.553495 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:14.553412 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 17:40:14.553495 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:14.553441 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 17:40:14.553495 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:14.553456 2561 projected.go:196] Error preparing data for projected volume kube-api-access-9gv8m for pod openshift-network-diagnostics/network-check-target-4q8f4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:40:14.553636 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:14.553510 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m podName:af9a2eb4-19e8-4a4c-a655-5fd0c7889235 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:22.553491508 +0000 UTC m=+18.166497358 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gv8m" (UniqueName: "kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m") pod "network-check-target-4q8f4" (UID: "af9a2eb4-19e8-4a4c-a655-5fd0c7889235") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:40:14.935963 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:14.935826 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:14.936133 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:14.935974 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:14.936439 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:14.936223 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:14.936439 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:14.936356 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:15.935303 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:15.935261 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:15.935789 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:15.935402 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:16.935595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:16.935561 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:16.936039 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:16.935697 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:16.936039 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:16.935906 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:16.936136 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:16.936056 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:17.935492 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:17.935461 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:17.935654 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:17.935588 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:18.935041 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:18.934953 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:18.935198 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:18.934955 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:18.935198 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:18.935075 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:18.935281 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:18.935203 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:19.934697 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:19.934655 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:19.935134 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:19.934775 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:20.935220 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:20.935184 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:20.935693 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:20.935184 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:20.935693 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:20.935336 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:20.935693 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:20.935391 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:21.935241 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:21.935209 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:21.935670 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:21.935342 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:22.519474 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:22.519440 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs\") pod \"network-metrics-daemon-2fs78\" (UID: \"fa9f496c-49f7-4a62-a5a9-1bb903fc553d\") " pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:22.519604 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:22.519488 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:22.519604 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:22.519574 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 17:40:22.519604 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:22.519575 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:40:22.519757 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:22.519626 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs podName:fa9f496c-49f7-4a62-a5a9-1bb903fc553d nodeName:}" failed. No retries permitted until 2026-07-24 17:40:38.519610697 +0000 UTC m=+34.132616540 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs") pod "network-metrics-daemon-2fs78" (UID: "fa9f496c-49f7-4a62-a5a9-1bb903fc553d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:40:22.519757 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:22.519643 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret podName:a3c435cc-32ae-4849-a167-4cf7d91b6832 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:38.519634254 +0000 UTC m=+34.132640097 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret") pod "global-pull-secret-syncer-zp4x2" (UID: "a3c435cc-32ae-4849-a167-4cf7d91b6832") : object "kube-system"/"original-pull-secret" not registered Jul 24 17:40:22.620812 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:22.620726 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gv8m\" (UniqueName: \"kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m\") pod \"network-check-target-4q8f4\" (UID: \"af9a2eb4-19e8-4a4c-a655-5fd0c7889235\") " pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:22.620998 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:22.620882 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 17:40:22.620998 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:22.620901 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 17:40:22.620998 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:22.620913 2561 projected.go:196] Error preparing data for projected volume kube-api-access-9gv8m for pod openshift-network-diagnostics/network-check-target-4q8f4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:40:22.620998 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:22.620994 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m podName:af9a2eb4-19e8-4a4c-a655-5fd0c7889235 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:38.620976519 +0000 UTC m=+34.233982367 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gv8m" (UniqueName: "kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m") pod "network-check-target-4q8f4" (UID: "af9a2eb4-19e8-4a4c-a655-5fd0c7889235") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:40:22.934742 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:22.934656 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:22.934898 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:22.934652 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:22.934898 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:22.934772 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:22.934898 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:22.934842 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:23.934909 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:23.934873 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:23.935305 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:23.935022 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:24.936130 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:24.936099 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:24.936572 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:24.936202 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:24.936572 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:24.936278 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:24.936572 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:24.936377 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:25.935121 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:25.935085 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:25.935319 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:25.935200 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:26.935206 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:26.934804 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:26.935783 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:26.934851 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:26.935783 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:26.935338 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:26.935783 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:26.935402 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:27.103989 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.103962 2561 generic.go:356] "Generic (PLEG): container finished" podID="397fd802-aa1f-4a3e-bd2b-6835089bbf23" containerID="1e85d8749d67a39c7e8b86795f3f5b6af4eda67a05e9fb27d4d4b121c2a20275" exitCode=0 Jul 24 17:40:27.104089 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.104050 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzfj6" event={"ID":"397fd802-aa1f-4a3e-bd2b-6835089bbf23","Type":"ContainerDied","Data":"1e85d8749d67a39c7e8b86795f3f5b6af4eda67a05e9fb27d4d4b121c2a20275"} Jul 24 17:40:27.105394 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.105348 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-28ktn" event={"ID":"a42c7507-5b58-40f6-ae1e-e42e1b578499","Type":"ContainerStarted","Data":"4b4e84ba60735e063f8aeb761bbf61fc702117c9816faac80993374b2517ec0d"} Jul 24 17:40:27.106812 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.106787 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5lrd6" event={"ID":"25eb2f6d-27ac-4259-bc89-fbcaa6cf085e","Type":"ContainerStarted","Data":"e70e07f1fa5efddcab631f9239a924fb57bee4266126534f25188458416f7be9"} Jul 24 17:40:27.108212 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.108192 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rdlc9" event={"ID":"e6d525b8-1f5d-4568-ae50-d71eb372ca67","Type":"ContainerStarted","Data":"52c1b6fe0eeefbef9d92f3b772d21b1ef255e18a1aaef1818b1358e80bfa7398"} Jul 24 17:40:27.110397 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.110380 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" event={"ID":"7c547f2c-359d-4d0e-b752-5ecf0c57872e","Type":"ContainerStarted","Data":"880ef8739fff33049eb6d70aabd5504a8c38963ea992151cfe8224e5909af51c"} Jul 24 17:40:27.110486 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.110436 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" event={"ID":"7c547f2c-359d-4d0e-b752-5ecf0c57872e","Type":"ContainerStarted","Data":"26ec87683b7b5914998acf0432ed21598ad716419b58fe6434b7c5edd0343b24"} Jul 24 17:40:27.110486 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.110482 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" event={"ID":"7c547f2c-359d-4d0e-b752-5ecf0c57872e","Type":"ContainerStarted","Data":"ec53437e3524c1e30267e29dfa39c937b398f430677489d30890e641db1221cc"} Jul 24 17:40:27.110581 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.110530 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" event={"ID":"7c547f2c-359d-4d0e-b752-5ecf0c57872e","Type":"ContainerStarted","Data":"3c77c1e2a2f51582b73902e6c9bb84b7294280e0e28b79560004ccd7e0044b8d"} Jul 24 17:40:27.111688 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.111670 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-72fk4" event={"ID":"5200e3bb-ae3f-4798-9909-0657616b6c7a","Type":"ContainerStarted","Data":"d05a767c09ae87a82a617e9dadb76361fd3538929713441da29cae10c355b16f"} Jul 24 17:40:27.112885 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.112862 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2r8wb" event={"ID":"525c91a9-d541-474e-9c6a-90cd9d3c460d","Type":"ContainerStarted","Data":"442bc51359dd7cf33b35c0e53d176599241de42664375c87418f2425faed0b61"} Jul 24 17:40:27.114095 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.114078 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" event={"ID":"265d5710-707a-4fab-8ddb-b253fc6a3e74","Type":"ContainerStarted","Data":"bb7bbf4af0aa8766f4feaf8ad1b1eaa3ba43853f10a274108401732d7e22d024"} Jul 24 17:40:27.130060 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.130015 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" podStartSLOduration=21.129999864 podStartE2EDuration="21.129999864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:40:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:10.112618144 +0000 UTC m=+5.725624015" watchObservedRunningTime="2026-07-24 17:40:27.129999864 +0000 UTC m=+22.743005733" Jul 24 17:40:27.146466 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.146429 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rdlc9" podStartSLOduration=3.630633374 podStartE2EDuration="22.146416827s" totalImagesPullingTime="18.515783453s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:27.146051006 +0000 UTC m=+22.759056872" watchObservedRunningTime="2026-07-24 17:40:27.146416827 +0000 UTC m=+22.759422693" Jul 24 17:40:27.168684 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.168643 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-72fk4" podStartSLOduration=3.606513416 podStartE2EDuration="22.168632698s" totalImagesPullingTime="18.562119282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:27.168462481 +0000 UTC m=+22.781468347" watchObservedRunningTime="2026-07-24 17:40:27.168632698 +0000 UTC m=+22.781638564" Jul 24 17:40:27.200033 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.199985 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-28ktn" podStartSLOduration=3.658717797 podStartE2EDuration="22.199971191s" totalImagesPullingTime="18.541253394s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:27.185436799 +0000 UTC m=+22.798442664" watchObservedRunningTime="2026-07-24 17:40:27.199971191 +0000 UTC m=+22.812977056" Jul 24 17:40:27.216174 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.216133 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2r8wb" podStartSLOduration=3.7046732909999998 podStartE2EDuration="22.216120037s" totalImagesPullingTime="18.511446746s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:27.199925509 +0000 UTC m=+22.812931374" watchObservedRunningTime="2026-07-24 17:40:27.216120037 +0000 UTC m=+22.829125903" Jul 24 17:40:27.216380 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.216359 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5lrd6" podStartSLOduration=7.92434926 podStartE2EDuration="22.216352676s" totalImagesPullingTime="14.292003416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:27.215399999 +0000 UTC m=+22.828405863" watchObservedRunningTime="2026-07-24 17:40:27.216352676 +0000 UTC m=+22.829358541" Jul 24 17:40:27.313470 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.313395 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:27.391456 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.391424 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:27.709397 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.709335 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:27.805791 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.805763 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:27.927068 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.927033 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:27.934833 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:27.934806 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:27.934989 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:27.934925 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:28.028133 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:28.028057 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:28.117411 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:28.117378 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f9brh" event={"ID":"b6d9717a-5dd0-410d-b016-9f56d775e96e","Type":"ContainerStarted","Data":"5c7c2b449ee88e198f96be967b77b635d3aa33d379a6402afbcd468f2d8163ef"} Jul 24 17:40:28.120845 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:28.120819 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" event={"ID":"7c547f2c-359d-4d0e-b752-5ecf0c57872e","Type":"ContainerStarted","Data":"a5ed82285d9526a923570d489a267d841a9736df80ab181753fd6bfc95d33ec6"} Jul 24 17:40:28.121004 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:28.120897 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" event={"ID":"7c547f2c-359d-4d0e-b752-5ecf0c57872e","Type":"ContainerStarted","Data":"ba0497f73d7341e37d2a8f8e5527e136855bf871eaf4ffecd4370fd7f21c54ac"} Jul 24 17:40:28.255456 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:28.255428 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:28.317058 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:28.317019 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 17:40:28.354874 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:28.354840 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:28.841394 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:28.841278 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T17:40:28.317040513Z","UUID":"ee31c3da-d562-4a2d-934d-09a5243ff776","Handler":null,"Name":"","Endpoint":""} Jul 24 17:40:28.842816 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:28.842795 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 17:40:28.843007 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:28.842840 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 17:40:28.935507 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:28.935473 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:28.935683 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:28.935615 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:28.935771 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:28.935752 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:28.935892 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:28.935870 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:29.125912 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:29.125825 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" event={"ID":"265d5710-707a-4fab-8ddb-b253fc6a3e74","Type":"ContainerStarted","Data":"27052715f2214910a34adb88a3a5d730860eb4ccc8ea574b438c2fa5fbf228c6"} Jul 24 17:40:29.399676 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:29.399581 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:29.497194 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:29.497152 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:29.597910 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:29.597876 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:29.657499 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:29.657415 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:29.935732 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:29.935500 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:29.935876 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:29.935789 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:30.132263 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:30.132222 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" event={"ID":"7c547f2c-359d-4d0e-b752-5ecf0c57872e","Type":"ContainerStarted","Data":"a561521d63e3213d0c3fbf263519bb1b0e26c5326d60f0ef6f6cdc7c6f05e5dd"} Jul 24 17:40:30.134527 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:30.134490 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" event={"ID":"265d5710-707a-4fab-8ddb-b253fc6a3e74","Type":"ContainerStarted","Data":"a21763ffc57fd4b1c15e7fa363228b4917ed52885811a2f2ceece8a79362dcc6"} Jul 24 17:40:30.158758 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:30.158701 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f9brh" podStartSLOduration=6.63677276 podStartE2EDuration="25.158683116s" totalImagesPullingTime="18.521910356s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:28.13419548 +0000 UTC m=+23.747201347" watchObservedRunningTime="2026-07-24 17:40:30.158683116 +0000 UTC m=+25.771688981" Jul 24 17:40:30.159106 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:30.159066 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4cdv" podStartSLOduration=3.775625142 podStartE2EDuration="25.15905527s" totalImagesPullingTime="21.383430128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:40:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:30.151933491 +0000 UTC m=+25.764939358" watchObservedRunningTime="2026-07-24 17:40:30.15905527 +0000 UTC m=+25.772061142" Jul 24 17:40:30.935480 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:30.935446 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:30.935697 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:30.935585 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:30.935697 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:30.935681 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:30.935808 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:30.935768 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:31.484110 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:31.484087 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5lrd6" Jul 24 17:40:31.484549 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:31.484535 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5lrd6" Jul 24 17:40:31.490439 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:31.490350 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5lrd6" Jul 24 17:40:31.490675 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:31.490660 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5lrd6" Jul 24 17:40:31.493951 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:31.493919 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:31.631169 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:31.631024 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:31.732762 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:31.732652 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:31.833339 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:31.833308 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:31.934741 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:31.934707 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:31.934903 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:31.934824 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:32.030889 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:32.030806 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:32.131167 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:32.131134 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:32.143083 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:32.143053 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" event={"ID":"7c547f2c-359d-4d0e-b752-5ecf0c57872e","Type":"ContainerStarted","Data":"304646725bef4736f6738f09859aacb0aad7e43dd5c76616d436407d86c3ee4e"} Jul 24 17:40:32.143328 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:32.143279 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:32.143404 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:32.143344 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:32.144977 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:32.144953 2561 generic.go:356] "Generic (PLEG): container finished" podID="397fd802-aa1f-4a3e-bd2b-6835089bbf23" containerID="f9ad7b2193b235a59a2051597518e8887586a1311ed7854fa5233b4b8af827e3" exitCode=0 Jul 24 17:40:32.145057 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:32.145025 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzfj6" event={"ID":"397fd802-aa1f-4a3e-bd2b-6835089bbf23","Type":"ContainerDied","Data":"f9ad7b2193b235a59a2051597518e8887586a1311ed7854fa5233b4b8af827e3"} Jul 24 17:40:32.159679 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:32.159658 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:32.170180 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:32.170141 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" podStartSLOduration=8.555270364 podStartE2EDuration="27.170128663s" totalImagesPullingTime="18.614858299s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:32.169878825 +0000 UTC m=+27.782884690" watchObservedRunningTime="2026-07-24 17:40:32.170128663 +0000 UTC m=+27.783134528" Jul 24 17:40:32.841043 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:32.841009 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:32.935558 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:32.935527 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:32.935558 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:32.935551 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:32.935758 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:32.935695 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:32.935874 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:32.935836 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:33.147504 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:33.147434 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:33.164648 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:33.164624 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:40:33.254767 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:33.254723 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:33.355256 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:33.355223 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:33.807212 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:33.807130 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zp4x2"] Jul 24 17:40:33.807347 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:33.807246 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:33.807397 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:33.807337 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:33.810247 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:33.810197 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4q8f4"] Jul 24 17:40:33.810366 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:33.810307 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:33.810428 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:33.810408 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:33.810733 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:33.810708 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2fs78"] Jul 24 17:40:33.810823 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:33.810811 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:33.810929 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:33.810909 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:34.151299 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:34.151208 2561 generic.go:356] "Generic (PLEG): container finished" podID="397fd802-aa1f-4a3e-bd2b-6835089bbf23" containerID="f95d0dacef03e16ba3709025aee16989480ca0ec7160eb24d290de81476af30a" exitCode=0 Jul 24 17:40:34.151299 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:34.151284 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzfj6" event={"ID":"397fd802-aa1f-4a3e-bd2b-6835089bbf23","Type":"ContainerDied","Data":"f95d0dacef03e16ba3709025aee16989480ca0ec7160eb24d290de81476af30a"} Jul 24 17:40:35.934529 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:35.934455 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:35.934529 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:35.934467 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:35.935097 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:35.934555 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:35.935097 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:35.934615 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:35.935097 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:35.934668 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:35.935097 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:35.934733 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:36.157603 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:36.157571 2561 generic.go:356] "Generic (PLEG): container finished" podID="397fd802-aa1f-4a3e-bd2b-6835089bbf23" containerID="df921795d0a348289f1e1377b2afddbc56544652704d34eb5268cc8ce24a0402" exitCode=0 Jul 24 17:40:36.157770 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:36.157637 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzfj6" event={"ID":"397fd802-aa1f-4a3e-bd2b-6835089bbf23","Type":"ContainerDied","Data":"df921795d0a348289f1e1377b2afddbc56544652704d34eb5268cc8ce24a0402"} Jul 24 17:40:36.517138 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:36.517050 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:37.935096 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:37.935068 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:37.935096 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:37.935093 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:37.935645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:37.935108 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:37.935645 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:37.935212 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:37.935645 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:37.935356 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:37.935645 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:37.935443 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:38.031812 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:38.031767 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:38.134038 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:38.134005 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:38.548531 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:38.548449 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs\") pod \"network-metrics-daemon-2fs78\" (UID: \"fa9f496c-49f7-4a62-a5a9-1bb903fc553d\") " pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:38.548531 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:38.548503 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:38.548722 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:38.548595 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:40:38.548722 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:38.548661 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs podName:fa9f496c-49f7-4a62-a5a9-1bb903fc553d nodeName:}" failed. No retries permitted until 2026-07-24 17:41:10.548646227 +0000 UTC m=+66.161652074 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs") pod "network-metrics-daemon-2fs78" (UID: "fa9f496c-49f7-4a62-a5a9-1bb903fc553d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:40:38.548722 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:38.548595 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 17:40:38.548868 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:38.548766 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret podName:a3c435cc-32ae-4849-a167-4cf7d91b6832 nodeName:}" failed. No retries permitted until 2026-07-24 17:41:10.548743692 +0000 UTC m=+66.161749536 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret") pod "global-pull-secret-syncer-zp4x2" (UID: "a3c435cc-32ae-4849-a167-4cf7d91b6832") : object "kube-system"/"original-pull-secret" not registered Jul 24 17:40:38.649334 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:38.649299 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gv8m\" (UniqueName: \"kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m\") pod \"network-check-target-4q8f4\" (UID: \"af9a2eb4-19e8-4a4c-a655-5fd0c7889235\") " pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:38.649499 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:38.649480 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 17:40:38.649499 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:38.649500 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 17:40:38.649598 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:38.649510 2561 projected.go:196] Error preparing data for projected volume kube-api-access-9gv8m for pod openshift-network-diagnostics/network-check-target-4q8f4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:40:38.649598 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:38.649563 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m podName:af9a2eb4-19e8-4a4c-a655-5fd0c7889235 nodeName:}" failed. No retries permitted until 2026-07-24 17:41:10.649548482 +0000 UTC m=+66.262554325 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gv8m" (UniqueName: "kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m") pod "network-check-target-4q8f4" (UID: "af9a2eb4-19e8-4a4c-a655-5fd0c7889235") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:40:38.792703 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:38.792667 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:38.861319 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:38.861196 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:38.945236 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:38.945159 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:39.181591 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:39.181555 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:39.241657 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:39.241579 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:39.623581 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:39.623495 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:39.718226 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:39.718193 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:39.935343 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:39.935319 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:39.935481 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:39.935316 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:39.935481 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:39.935327 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:39.935481 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:39.935429 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q8f4" podUID="af9a2eb4-19e8-4a4c-a655-5fd0c7889235" Jul 24 17:40:39.935619 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:39.935552 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fs78" podUID="fa9f496c-49f7-4a62-a5a9-1bb903fc553d" Jul 24 17:40:39.935663 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:39.935633 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zp4x2" podUID="a3c435cc-32ae-4849-a167-4cf7d91b6832" Jul 24 17:40:39.951694 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:39.951671 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rdlc9_e6d525b8-1f5d-4568-ae50-d71eb372ca67/dns-node-resolver/0.log" Jul 24 17:40:40.159585 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.159558 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeReady" Jul 24 17:40:40.159753 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.159712 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 17:40:40.206293 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.206222 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69d8678b4b-sj7rh"] Jul 24 17:40:40.211082 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.211059 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.214679 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.214654 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 24 17:40:40.214862 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.214823 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 24 17:40:40.214862 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.214848 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-crf7b\"" Jul 24 17:40:40.215409 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.215391 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 24 17:40:40.223019 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.222843 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 24 17:40:40.227561 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.227541 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69d8678b4b-sj7rh"] Jul 24 17:40:40.236145 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.236122 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tmsv8"] Jul 24 17:40:40.240862 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.240817 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zcvr7"] Jul 24 17:40:40.241307 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.241277 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:40.244636 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.244612 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.247599 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.247579 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 17:40:40.247780 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.247762 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 17:40:40.247861 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.247819 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 17:40:40.248101 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.248056 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 17:40:40.248101 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.248078 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 17:40:40.248246 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.248102 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 17:40:40.248246 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.248129 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-g9gf7\"" Jul 24 17:40:40.248246 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.248079 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tq4qz\"" Jul 24 17:40:40.256505 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.256187 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tmsv8"] Jul 24 17:40:40.256983 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.256961 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zcvr7"] Jul 24 17:40:40.265722 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.265700 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lxj4f"] Jul 24 17:40:40.268841 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.268818 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lxj4f" Jul 24 17:40:40.271714 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.271694 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 17:40:40.271868 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.271844 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 17:40:40.271969 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.271954 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jpmgf\"" Jul 24 17:40:40.272011 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.271991 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 17:40:40.279816 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.279796 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lxj4f"] Jul 24 17:40:40.362088 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362057 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzkcb\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-kube-api-access-gzkcb\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.362266 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362108 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.362266 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362131 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aa20ebee-6777-4053-82f0-6add3bb8b26e-ca-trust-extracted\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.362266 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362179 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.362266 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362201 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.362266 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362224 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aa20ebee-6777-4053-82f0-6add3bb8b26e-trusted-ca\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.362266 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362243 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aa20ebee-6777-4053-82f0-6add3bb8b26e-installation-pull-secrets\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.362266 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362267 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-bound-sa-token\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.362575 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362290 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/21676a3c-bd55-4e4a-ab8e-4275108751f2-config-volume\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:40.362575 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362333 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/afa66580-b192-471f-a0a1-2eadd6d85817-crio-socket\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.362575 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362390 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrrb5\" (UniqueName: \"kubernetes.io/projected/afa66580-b192-471f-a0a1-2eadd6d85817-kube-api-access-nrrb5\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.362575 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362461 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aa20ebee-6777-4053-82f0-6add3bb8b26e-image-registry-private-configuration\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.362575 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362485 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9kf2\" (UniqueName: \"kubernetes.io/projected/21676a3c-bd55-4e4a-ab8e-4275108751f2-kube-api-access-d9kf2\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:40.362575 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362547 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:40.362575 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362576 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21676a3c-bd55-4e4a-ab8e-4275108751f2-tmp-dir\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:40.362840 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362616 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-certificates\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.362840 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.362637 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/afa66580-b192-471f-a0a1-2eadd6d85817-data-volume\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.401569 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.401507 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:40.459106 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.459026 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:40.462912 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.462883 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.463042 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.462916 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aa20ebee-6777-4053-82f0-6add3bb8b26e-ca-trust-extracted\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.463042 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.462953 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.463042 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.462979 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.463042 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.462994 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aa20ebee-6777-4053-82f0-6add3bb8b26e-trusted-ca\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.463042 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.463009 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aa20ebee-6777-4053-82f0-6add3bb8b26e-installation-pull-secrets\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.463042 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.463020 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 17:40:40.463338 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.463143 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm podName:afa66580-b192-471f-a0a1-2eadd6d85817 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:40.963121676 +0000 UTC m=+36.576127540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zcvr7" (UID: "afa66580-b192-471f-a0a1-2eadd6d85817") : configmap "kube-rbac-proxy" not found Jul 24 17:40:40.463338 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.463315 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 17:40:40.463452 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.463340 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aa20ebee-6777-4053-82f0-6add3bb8b26e-ca-trust-extracted\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.463452 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.463028 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-bound-sa-token\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.463452 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.463359 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls podName:afa66580-b192-471f-a0a1-2eadd6d85817 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:40.963347863 +0000 UTC m=+36.576353707 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zcvr7" (UID: "afa66580-b192-471f-a0a1-2eadd6d85817") : secret "insights-runtime-extractor-tls" not found Jul 24 17:40:40.463452 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.463402 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/21676a3c-bd55-4e4a-ab8e-4275108751f2-config-volume\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:40.463452 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.463445 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/afa66580-b192-471f-a0a1-2eadd6d85817-crio-socket\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.463698 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.463470 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrrb5\" (UniqueName: \"kubernetes.io/projected/afa66580-b192-471f-a0a1-2eadd6d85817-kube-api-access-nrrb5\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.463698 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.463497 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aa20ebee-6777-4053-82f0-6add3bb8b26e-image-registry-private-configuration\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.463698 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.463523 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9kf2\" (UniqueName: \"kubernetes.io/projected/21676a3c-bd55-4e4a-ab8e-4275108751f2-kube-api-access-d9kf2\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:40.463698 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.463551 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e12eb736-035d-481f-afcf-9a09118b554a-cert\") pod \"ingress-canary-lxj4f\" (UID: \"e12eb736-035d-481f-afcf-9a09118b554a\") " pod="openshift-ingress-canary/ingress-canary-lxj4f" Jul 24 17:40:40.463698 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.463593 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:40.463698 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.463624 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21676a3c-bd55-4e4a-ab8e-4275108751f2-tmp-dir\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:40.463698 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.463657 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdmbw\" (UniqueName: \"kubernetes.io/projected/e12eb736-035d-481f-afcf-9a09118b554a-kube-api-access-gdmbw\") pod \"ingress-canary-lxj4f\" (UID: \"e12eb736-035d-481f-afcf-9a09118b554a\") " pod="openshift-ingress-canary/ingress-canary-lxj4f" Jul 24 17:40:40.463698 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.463694 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-certificates\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.464084 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.463728 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/afa66580-b192-471f-a0a1-2eadd6d85817-data-volume\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.464084 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.463783 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzkcb\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-kube-api-access-gzkcb\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.464084 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.463808 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 17:40:40.464084 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.463823 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d8678b4b-sj7rh: secret "image-registry-tls" not found Jul 24 17:40:40.464084 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.463865 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls podName:aa20ebee-6777-4053-82f0-6add3bb8b26e nodeName:}" failed. No retries permitted until 2026-07-24 17:40:40.96384953 +0000 UTC m=+36.576855394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls") pod "image-registry-69d8678b4b-sj7rh" (UID: "aa20ebee-6777-4053-82f0-6add3bb8b26e") : secret "image-registry-tls" not found Jul 24 17:40:40.464084 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.464041 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 17:40:40.464415 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.464104 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls podName:21676a3c-bd55-4e4a-ab8e-4275108751f2 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:40.964086031 +0000 UTC m=+36.577091878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls") pod "dns-default-tmsv8" (UID: "21676a3c-bd55-4e4a-ab8e-4275108751f2") : secret "dns-default-metrics-tls" not found Jul 24 17:40:40.464415 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.464330 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aa20ebee-6777-4053-82f0-6add3bb8b26e-trusted-ca\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.464415 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.464385 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/21676a3c-bd55-4e4a-ab8e-4275108751f2-config-volume\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:40.464415 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.464400 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21676a3c-bd55-4e4a-ab8e-4275108751f2-tmp-dir\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:40.464646 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.464575 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/afa66580-b192-471f-a0a1-2eadd6d85817-crio-socket\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.464646 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.464596 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-certificates\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.464746 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.464642 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/afa66580-b192-471f-a0a1-2eadd6d85817-data-volume\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.468989 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.468953 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aa20ebee-6777-4053-82f0-6add3bb8b26e-installation-pull-secrets\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.468989 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.468977 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aa20ebee-6777-4053-82f0-6add3bb8b26e-image-registry-private-configuration\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.472065 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.472044 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzkcb\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-kube-api-access-gzkcb\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.472065 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.472054 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-bound-sa-token\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.472347 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.472327 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrrb5\" (UniqueName: \"kubernetes.io/projected/afa66580-b192-471f-a0a1-2eadd6d85817-kube-api-access-nrrb5\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.472752 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.472727 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9kf2\" (UniqueName: \"kubernetes.io/projected/21676a3c-bd55-4e4a-ab8e-4275108751f2-kube-api-access-d9kf2\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:40.565079 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.565045 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdmbw\" (UniqueName: \"kubernetes.io/projected/e12eb736-035d-481f-afcf-9a09118b554a-kube-api-access-gdmbw\") pod \"ingress-canary-lxj4f\" (UID: \"e12eb736-035d-481f-afcf-9a09118b554a\") " pod="openshift-ingress-canary/ingress-canary-lxj4f" Jul 24 17:40:40.565268 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.565200 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e12eb736-035d-481f-afcf-9a09118b554a-cert\") pod \"ingress-canary-lxj4f\" (UID: \"e12eb736-035d-481f-afcf-9a09118b554a\") " pod="openshift-ingress-canary/ingress-canary-lxj4f" Jul 24 17:40:40.567885 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.567859 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e12eb736-035d-481f-afcf-9a09118b554a-cert\") pod \"ingress-canary-lxj4f\" (UID: \"e12eb736-035d-481f-afcf-9a09118b554a\") " pod="openshift-ingress-canary/ingress-canary-lxj4f" Jul 24 17:40:40.578466 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.578447 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdmbw\" (UniqueName: \"kubernetes.io/projected/e12eb736-035d-481f-afcf-9a09118b554a-kube-api-access-gdmbw\") pod \"ingress-canary-lxj4f\" (UID: \"e12eb736-035d-481f-afcf-9a09118b554a\") " pod="openshift-ingress-canary/ingress-canary-lxj4f" Jul 24 17:40:40.587128 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.587105 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-lpw5h"] Jul 24 17:40:40.591800 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.591782 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lpw5h" Jul 24 17:40:40.593965 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.593925 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 24 17:40:40.594146 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.593974 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 24 17:40:40.594146 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.594107 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-9bpkt\"" Jul 24 17:40:40.597982 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.597959 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-lpw5h"] Jul 24 17:40:40.622675 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.622647 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:40.723735 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.723654 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:40.752462 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.752429 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2r8wb_525c91a9-d541-474e-9c6a-90cd9d3c460d/node-ca/0.log" Jul 24 17:40:40.766732 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.766700 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppf6v\" (UniqueName: \"kubernetes.io/projected/4944123d-acba-498e-9140-a1645675f417-kube-api-access-ppf6v\") pod \"migrator-6865c9967-lpw5h\" (UID: \"4944123d-acba-498e-9140-a1645675f417\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lpw5h" Jul 24 17:40:40.867513 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.867475 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppf6v\" (UniqueName: \"kubernetes.io/projected/4944123d-acba-498e-9140-a1645675f417-kube-api-access-ppf6v\") pod \"migrator-6865c9967-lpw5h\" (UID: \"4944123d-acba-498e-9140-a1645675f417\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lpw5h" Jul 24 17:40:40.876011 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.875978 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppf6v\" (UniqueName: \"kubernetes.io/projected/4944123d-acba-498e-9140-a1645675f417-kube-api-access-ppf6v\") pod \"migrator-6865c9967-lpw5h\" (UID: \"4944123d-acba-498e-9140-a1645675f417\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lpw5h" Jul 24 17:40:40.877770 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.877752 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lxj4f" Jul 24 17:40:40.903275 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.903252 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lpw5h" Jul 24 17:40:40.967903 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.967864 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:40.968244 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.968084 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.968244 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.968126 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:40.968244 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:40.968172 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:40.968354 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.968338 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 17:40:40.968391 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.968354 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d8678b4b-sj7rh: secret "image-registry-tls" not found Jul 24 17:40:40.968422 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.968413 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls podName:aa20ebee-6777-4053-82f0-6add3bb8b26e nodeName:}" failed. No retries permitted until 2026-07-24 17:40:41.968393896 +0000 UTC m=+37.581399742 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls") pod "image-registry-69d8678b4b-sj7rh" (UID: "aa20ebee-6777-4053-82f0-6add3bb8b26e") : secret "image-registry-tls" not found Jul 24 17:40:40.968530 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.968516 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 17:40:40.968572 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.968563 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm podName:afa66580-b192-471f-a0a1-2eadd6d85817 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:41.968550725 +0000 UTC m=+37.581556579 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zcvr7" (UID: "afa66580-b192-471f-a0a1-2eadd6d85817") : configmap "kube-rbac-proxy" not found Jul 24 17:40:40.968666 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.968634 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 17:40:40.968734 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.968667 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls podName:21676a3c-bd55-4e4a-ab8e-4275108751f2 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:41.968657734 +0000 UTC m=+37.581663593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls") pod "dns-default-tmsv8" (UID: "21676a3c-bd55-4e4a-ab8e-4275108751f2") : secret "dns-default-metrics-tls" not found Jul 24 17:40:40.968734 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.968714 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 17:40:40.968834 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:40.968780 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls podName:afa66580-b192-471f-a0a1-2eadd6d85817 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:41.968760119 +0000 UTC m=+37.581765964 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zcvr7" (UID: "afa66580-b192-471f-a0a1-2eadd6d85817") : secret "insights-runtime-extractor-tls" not found Jul 24 17:40:41.028707 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.028675 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lxj4f"] Jul 24 17:40:41.043030 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.043004 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-lpw5h"] Jul 24 17:40:41.046124 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:41.046091 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode12eb736_035d_481f_afcf_9a09118b554a.slice/crio-b8a7cead9caeb630a4ebc53cbe4272e596b102b7b99d726278bc1e2fe63050b0 WatchSource:0}: Error finding container b8a7cead9caeb630a4ebc53cbe4272e596b102b7b99d726278bc1e2fe63050b0: Status 404 returned error can't find the container with id b8a7cead9caeb630a4ebc53cbe4272e596b102b7b99d726278bc1e2fe63050b0 Jul 24 17:40:41.046686 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:41.046659 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4944123d_acba_498e_9140_a1645675f417.slice/crio-f3451adfb86a16bd2695a3c4dc2c98ab41671d954826c8e961f1d7f4a920d355 WatchSource:0}: Error finding container f3451adfb86a16bd2695a3c4dc2c98ab41671d954826c8e961f1d7f4a920d355: Status 404 returned error can't find the container with id f3451adfb86a16bd2695a3c4dc2c98ab41671d954826c8e961f1d7f4a920d355 Jul 24 17:40:41.169347 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.169319 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lpw5h" event={"ID":"4944123d-acba-498e-9140-a1645675f417","Type":"ContainerStarted","Data":"f3451adfb86a16bd2695a3c4dc2c98ab41671d954826c8e961f1d7f4a920d355"} Jul 24 17:40:41.170335 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.170305 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lxj4f" event={"ID":"e12eb736-035d-481f-afcf-9a09118b554a","Type":"ContainerStarted","Data":"b8a7cead9caeb630a4ebc53cbe4272e596b102b7b99d726278bc1e2fe63050b0"} Jul 24 17:40:41.172947 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.172910 2561 generic.go:356] "Generic (PLEG): container finished" podID="397fd802-aa1f-4a3e-bd2b-6835089bbf23" containerID="5105e9d582ce2675d37ddc407c897c830e3362fc21548833fec5bce0a4b17a82" exitCode=0 Jul 24 17:40:41.173058 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.172958 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzfj6" event={"ID":"397fd802-aa1f-4a3e-bd2b-6835089bbf23","Type":"ContainerDied","Data":"5105e9d582ce2675d37ddc407c897c830e3362fc21548833fec5bce0a4b17a82"} Jul 24 17:40:41.246125 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.246102 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-25b8d"] Jul 24 17:40:41.276441 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.276293 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-25b8d"] Jul 24 17:40:41.276524 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.276405 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-25b8d" Jul 24 17:40:41.278820 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.278793 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 24 17:40:41.278960 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.278837 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 24 17:40:41.278960 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.278888 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-lmfg6\"" Jul 24 17:40:41.278960 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.278892 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 24 17:40:41.279097 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.278895 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 24 17:40:41.283951 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.283921 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:41.370391 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.370367 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/551d2de0-6b59-43b5-841b-1c719f087f7d-signing-cabundle\") pod \"service-ca-658d7b7c78-25b8d\" (UID: \"551d2de0-6b59-43b5-841b-1c719f087f7d\") " pod="openshift-service-ca/service-ca-658d7b7c78-25b8d" Jul 24 17:40:41.370523 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.370410 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/551d2de0-6b59-43b5-841b-1c719f087f7d-signing-key\") pod \"service-ca-658d7b7c78-25b8d\" (UID: \"551d2de0-6b59-43b5-841b-1c719f087f7d\") " pod="openshift-service-ca/service-ca-658d7b7c78-25b8d" Jul 24 17:40:41.370616 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.370597 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s76vs\" (UniqueName: \"kubernetes.io/projected/551d2de0-6b59-43b5-841b-1c719f087f7d-kube-api-access-s76vs\") pod \"service-ca-658d7b7c78-25b8d\" (UID: \"551d2de0-6b59-43b5-841b-1c719f087f7d\") " pod="openshift-service-ca/service-ca-658d7b7c78-25b8d" Jul 24 17:40:41.471089 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.471060 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s76vs\" (UniqueName: \"kubernetes.io/projected/551d2de0-6b59-43b5-841b-1c719f087f7d-kube-api-access-s76vs\") pod \"service-ca-658d7b7c78-25b8d\" (UID: \"551d2de0-6b59-43b5-841b-1c719f087f7d\") " pod="openshift-service-ca/service-ca-658d7b7c78-25b8d" Jul 24 17:40:41.471189 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.471100 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/551d2de0-6b59-43b5-841b-1c719f087f7d-signing-cabundle\") pod \"service-ca-658d7b7c78-25b8d\" (UID: \"551d2de0-6b59-43b5-841b-1c719f087f7d\") " pod="openshift-service-ca/service-ca-658d7b7c78-25b8d" Jul 24 17:40:41.471189 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.471135 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/551d2de0-6b59-43b5-841b-1c719f087f7d-signing-key\") pod \"service-ca-658d7b7c78-25b8d\" (UID: \"551d2de0-6b59-43b5-841b-1c719f087f7d\") " pod="openshift-service-ca/service-ca-658d7b7c78-25b8d" Jul 24 17:40:41.471870 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.471841 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/551d2de0-6b59-43b5-841b-1c719f087f7d-signing-cabundle\") pod \"service-ca-658d7b7c78-25b8d\" (UID: \"551d2de0-6b59-43b5-841b-1c719f087f7d\") " pod="openshift-service-ca/service-ca-658d7b7c78-25b8d" Jul 24 17:40:41.474615 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.474586 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/551d2de0-6b59-43b5-841b-1c719f087f7d-signing-key\") pod \"service-ca-658d7b7c78-25b8d\" (UID: \"551d2de0-6b59-43b5-841b-1c719f087f7d\") " pod="openshift-service-ca/service-ca-658d7b7c78-25b8d" Jul 24 17:40:41.479106 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.479087 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s76vs\" (UniqueName: \"kubernetes.io/projected/551d2de0-6b59-43b5-841b-1c719f087f7d-kube-api-access-s76vs\") pod \"service-ca-658d7b7c78-25b8d\" (UID: \"551d2de0-6b59-43b5-841b-1c719f087f7d\") " pod="openshift-service-ca/service-ca-658d7b7c78-25b8d" Jul 24 17:40:41.584350 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.584270 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-25b8d" Jul 24 17:40:41.745533 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.745503 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-25b8d"] Jul 24 17:40:41.749132 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:41.749103 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod551d2de0_6b59_43b5_841b_1c719f087f7d.slice/crio-2317c3ed01d224b16a1d5acd5a5814dc19c297d29068e05e78652b2dabbb46c7 WatchSource:0}: Error finding container 2317c3ed01d224b16a1d5acd5a5814dc19c297d29068e05e78652b2dabbb46c7: Status 404 returned error can't find the container with id 2317c3ed01d224b16a1d5acd5a5814dc19c297d29068e05e78652b2dabbb46c7 Jul 24 17:40:41.935000 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.934977 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:40:41.935131 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.935007 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:40:41.935131 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.935016 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:40:41.937571 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.937546 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 17:40:41.937690 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.937590 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 17:40:41.937690 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.937598 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t8nmh\"" Jul 24 17:40:41.937690 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.937601 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-62mvz\"" Jul 24 17:40:41.937840 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.937706 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 17:40:41.938300 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.938282 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 17:40:41.976465 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.975580 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:41.976465 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.975639 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:41.976465 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.975686 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:41.976465 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:41.975748 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:41.976465 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:41.975934 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 17:40:41.976465 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:41.976084 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 17:40:41.976465 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:41.976119 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 17:40:41.976465 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:41.976137 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm podName:afa66580-b192-471f-a0a1-2eadd6d85817 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:43.976118 +0000 UTC m=+39.589123844 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zcvr7" (UID: "afa66580-b192-471f-a0a1-2eadd6d85817") : configmap "kube-rbac-proxy" not found Jul 24 17:40:41.976465 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:41.976178 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls podName:afa66580-b192-471f-a0a1-2eadd6d85817 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:43.976161291 +0000 UTC m=+39.589167141 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zcvr7" (UID: "afa66580-b192-471f-a0a1-2eadd6d85817") : secret "insights-runtime-extractor-tls" not found Jul 24 17:40:41.976465 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:41.976252 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 17:40:41.976465 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:41.976263 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d8678b4b-sj7rh: secret "image-registry-tls" not found Jul 24 17:40:41.976465 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:41.976298 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls podName:aa20ebee-6777-4053-82f0-6add3bb8b26e nodeName:}" failed. No retries permitted until 2026-07-24 17:40:43.976285516 +0000 UTC m=+39.589291364 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls") pod "image-registry-69d8678b4b-sj7rh" (UID: "aa20ebee-6777-4053-82f0-6add3bb8b26e") : secret "image-registry-tls" not found Jul 24 17:40:41.977379 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:41.976554 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls podName:21676a3c-bd55-4e4a-ab8e-4275108751f2 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:43.976538507 +0000 UTC m=+39.589544350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls") pod "dns-default-tmsv8" (UID: "21676a3c-bd55-4e4a-ab8e-4275108751f2") : secret "dns-default-metrics-tls" not found Jul 24 17:40:42.156696 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:42.156535 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:42.180123 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:42.180090 2561 generic.go:356] "Generic (PLEG): container finished" podID="397fd802-aa1f-4a3e-bd2b-6835089bbf23" containerID="2fc248b3c8683665ff85363af9e4a71186fb879b35c87bd51c492fce2ec9c3df" exitCode=0 Jul 24 17:40:42.180255 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:42.180169 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzfj6" event={"ID":"397fd802-aa1f-4a3e-bd2b-6835089bbf23","Type":"ContainerDied","Data":"2fc248b3c8683665ff85363af9e4a71186fb879b35c87bd51c492fce2ec9c3df"} Jul 24 17:40:42.181853 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:42.181823 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-25b8d" event={"ID":"551d2de0-6b59-43b5-841b-1c719f087f7d","Type":"ContainerStarted","Data":"2317c3ed01d224b16a1d5acd5a5814dc19c297d29068e05e78652b2dabbb46c7"} Jul 24 17:40:43.186112 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:43.185852 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lpw5h" event={"ID":"4944123d-acba-498e-9140-a1645675f417","Type":"ContainerStarted","Data":"8b3431a0739030961572af5574e8da088656a31a69f11035e0ef92b80a24078b"} Jul 24 17:40:43.186808 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:43.186148 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lpw5h" event={"ID":"4944123d-acba-498e-9140-a1645675f417","Type":"ContainerStarted","Data":"c9b6a21ebc281a45fdcfe7f68e71ba9ae5c4674cbac3315c285e88a15272706c"} Jul 24 17:40:43.189612 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:43.189583 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzfj6" event={"ID":"397fd802-aa1f-4a3e-bd2b-6835089bbf23","Type":"ContainerStarted","Data":"602f25e32fe992ca1b582af7fc9ed0e238e27dcd343197c6ee85713c863ea2bc"} Jul 24 17:40:43.202415 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:43.202360 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lpw5h" podStartSLOduration=1.684799655 podStartE2EDuration="3.202338664s" totalImagesPullingTime="1.517539009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:43.201237745 +0000 UTC m=+38.814243624" watchObservedRunningTime="2026-07-24 17:40:43.202338664 +0000 UTC m=+38.815344531" Jul 24 17:40:43.224224 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:43.224180 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fzfj6" podStartSLOduration=8.420889376 podStartE2EDuration="38.224165306s" totalImagesPullingTime="29.80327593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:40:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:43.222665986 +0000 UTC m=+38.835671865" watchObservedRunningTime="2026-07-24 17:40:43.224165306 +0000 UTC m=+38.837171172" Jul 24 17:40:43.992184 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:43.992153 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:43.992361 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:43.992197 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:43.992361 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:43.992227 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:43.992361 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:43.992281 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:43.992361 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:43.992291 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 17:40:43.992361 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:43.992322 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 17:40:43.992592 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:43.992370 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm podName:afa66580-b192-471f-a0a1-2eadd6d85817 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:47.992346103 +0000 UTC m=+43.605351972 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zcvr7" (UID: "afa66580-b192-471f-a0a1-2eadd6d85817") : configmap "kube-rbac-proxy" not found Jul 24 17:40:43.992592 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:43.992387 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls podName:afa66580-b192-471f-a0a1-2eadd6d85817 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:47.992380517 +0000 UTC m=+43.605386361 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zcvr7" (UID: "afa66580-b192-471f-a0a1-2eadd6d85817") : secret "insights-runtime-extractor-tls" not found Jul 24 17:40:43.992592 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:43.992399 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 17:40:43.992592 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:43.992421 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d8678b4b-sj7rh: secret "image-registry-tls" not found Jul 24 17:40:43.992592 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:43.992407 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 17:40:43.992592 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:43.992464 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls podName:21676a3c-bd55-4e4a-ab8e-4275108751f2 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:47.992452023 +0000 UTC m=+43.605457876 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls") pod "dns-default-tmsv8" (UID: "21676a3c-bd55-4e4a-ab8e-4275108751f2") : secret "dns-default-metrics-tls" not found Jul 24 17:40:43.992592 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:43.992479 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls podName:aa20ebee-6777-4053-82f0-6add3bb8b26e nodeName:}" failed. No retries permitted until 2026-07-24 17:40:47.992471318 +0000 UTC m=+43.605477162 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls") pod "image-registry-69d8678b4b-sj7rh" (UID: "aa20ebee-6777-4053-82f0-6add3bb8b26e") : secret "image-registry-tls" not found Jul 24 17:40:44.709074 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:44.709040 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:44.781889 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:44.781808 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:44.980371 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:44.980341 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:45.196234 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:45.196201 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-25b8d" event={"ID":"551d2de0-6b59-43b5-841b-1c719f087f7d","Type":"ContainerStarted","Data":"57f0bde6450f190245ef1be747ae67f9527d2803cdfca96d4d299b87a7648658"} Jul 24 17:40:45.197624 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:45.197597 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lxj4f" event={"ID":"e12eb736-035d-481f-afcf-9a09118b554a","Type":"ContainerStarted","Data":"fd557d0ee4f5c2f52d102893e6701afab06511354c6d35c5b3ce9d4c67bbf27b"} Jul 24 17:40:45.211226 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:45.211186 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-25b8d" podStartSLOduration=1.41501008 podStartE2EDuration="4.211168463s" totalImagesPullingTime="2.796158383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:45.210867003 +0000 UTC m=+40.823872863" watchObservedRunningTime="2026-07-24 17:40:45.211168463 +0000 UTC m=+40.824174330" Jul 24 17:40:45.226007 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:45.225969 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lxj4f" podStartSLOduration=1.731826051 podStartE2EDuration="5.225932315s" totalImagesPullingTime="3.494106264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:45.225466613 +0000 UTC m=+40.838472479" watchObservedRunningTime="2026-07-24 17:40:45.225932315 +0000 UTC m=+40.838938181" Jul 24 17:40:45.813701 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:45.813666 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:45.980436 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:45.980401 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:46.083199 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:46.083122 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:46.185127 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:46.185089 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:46.281458 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:46.281428 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:46.919153 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:46.919124 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:47.154765 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:47.154733 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:47.317667 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:47.317627 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:47.714304 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:47.714271 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:47.914984 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:47.914956 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:48.025948 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:48.025857 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:48.026368 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:48.025987 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:48.026368 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:48.026009 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 17:40:48.026368 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:48.026024 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:48.026368 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:48.026051 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:48.026368 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:48.026072 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls podName:21676a3c-bd55-4e4a-ab8e-4275108751f2 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:56.026051988 +0000 UTC m=+51.639057833 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls") pod "dns-default-tmsv8" (UID: "21676a3c-bd55-4e4a-ab8e-4275108751f2") : secret "dns-default-metrics-tls" not found Jul 24 17:40:48.026368 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:48.026116 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 17:40:48.026368 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:48.026138 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 17:40:48.026368 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:48.026172 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 17:40:48.026368 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:48.026185 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d8678b4b-sj7rh: secret "image-registry-tls" not found Jul 24 17:40:48.026368 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:48.026174 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls podName:afa66580-b192-471f-a0a1-2eadd6d85817 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:56.026161838 +0000 UTC m=+51.639167693 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zcvr7" (UID: "afa66580-b192-471f-a0a1-2eadd6d85817") : secret "insights-runtime-extractor-tls" not found Jul 24 17:40:48.026368 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:48.026240 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm podName:afa66580-b192-471f-a0a1-2eadd6d85817 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:56.026224372 +0000 UTC m=+51.639230223 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zcvr7" (UID: "afa66580-b192-471f-a0a1-2eadd6d85817") : configmap "kube-rbac-proxy" not found Jul 24 17:40:48.026368 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:40:48.026253 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls podName:aa20ebee-6777-4053-82f0-6add3bb8b26e nodeName:}" failed. No retries permitted until 2026-07-24 17:40:56.026244774 +0000 UTC m=+51.639250625 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls") pod "image-registry-69d8678b4b-sj7rh" (UID: "aa20ebee-6777-4053-82f0-6add3bb8b26e") : secret "image-registry-tls" not found Jul 24 17:40:48.310338 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:48.310249 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:48.453466 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:48.453432 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:48.554432 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:48.554387 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:48.707863 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:48.707836 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:48.909772 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:48.909739 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:49.313809 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:49.313767 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:49.510695 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:49.510663 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:49.720063 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:49.720018 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:49.779034 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:49.779001 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:49.910217 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:49.910180 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:50.108279 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:50.108204 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:50.172872 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:50.172833 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:51.108070 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:51.108041 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:51.506079 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:51.506042 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:51.711260 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:51.711232 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:51.915120 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:51.915029 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:52.122437 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:52.122405 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:52.172979 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:52.172892 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:53.308650 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:53.308615 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:53.516442 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:53.516401 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:53.617200 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:53.617116 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:53.717181 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:53.717144 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:53.913531 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:53.913451 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:54.108741 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:54.108699 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:55.116410 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:55.116376 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:55.313446 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:55.313418 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:55.514316 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:55.514281 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:55.716891 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:55.716853 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:56.090101 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.090068 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:56.090282 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.090105 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:56.090282 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.090126 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:56.090282 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.090162 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:56.090735 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.090713 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afa66580-b192-471f-a0a1-2eadd6d85817-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:56.093413 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.093391 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21676a3c-bd55-4e4a-ab8e-4275108751f2-metrics-tls\") pod \"dns-default-tmsv8\" (UID: \"21676a3c-bd55-4e4a-ab8e-4275108751f2\") " pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:56.093561 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.093539 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afa66580-b192-471f-a0a1-2eadd6d85817-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcvr7\" (UID: \"afa66580-b192-471f-a0a1-2eadd6d85817\") " pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:56.093660 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.093643 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls\") pod \"image-registry-69d8678b4b-sj7rh\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:56.122473 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.122450 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:56.152075 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.152052 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:56.159195 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.159173 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zcvr7" Jul 24 17:40:56.268679 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.268651 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69d8678b4b-sj7rh"] Jul 24 17:40:56.272348 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:56.272318 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa20ebee_6777_4053_82f0_6add3bb8b26e.slice/crio-6b8af7af37698afcc42d159818717ec67ef4c027e9cd10ca3aba718f191cfa50 WatchSource:0}: Error finding container 6b8af7af37698afcc42d159818717ec67ef4c027e9cd10ca3aba718f191cfa50: Status 404 returned error can't find the container with id 6b8af7af37698afcc42d159818717ec67ef4c027e9cd10ca3aba718f191cfa50 Jul 24 17:40:56.307228 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.307205 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tmsv8"] Jul 24 17:40:56.311157 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:56.311132 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21676a3c_bd55_4e4a_ab8e_4275108751f2.slice/crio-62fee178b210a3de68086f605396f7a8e6cbc7a7144a6595515bc88a4cdf71d5 WatchSource:0}: Error finding container 62fee178b210a3de68086f605396f7a8e6cbc7a7144a6595515bc88a4cdf71d5: Status 404 returned error can't find the container with id 62fee178b210a3de68086f605396f7a8e6cbc7a7144a6595515bc88a4cdf71d5 Jul 24 17:40:56.313798 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.313770 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:56.323712 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.323679 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zcvr7"] Jul 24 17:40:56.326342 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:40:56.326319 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafa66580_b192_471f_a0a1_2eadd6d85817.slice/crio-ac6bab58a1d14577c704377a6988a8c721bb17c8af81f298ca4be6e1bb8acda1 WatchSource:0}: Error finding container ac6bab58a1d14577c704377a6988a8c721bb17c8af81f298ca4be6e1bb8acda1: Status 404 returned error can't find the container with id ac6bab58a1d14577c704377a6988a8c721bb17c8af81f298ca4be6e1bb8acda1 Jul 24 17:40:56.519079 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:56.519045 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:40:57.231312 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:57.231267 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tmsv8" event={"ID":"21676a3c-bd55-4e4a-ab8e-4275108751f2","Type":"ContainerStarted","Data":"62fee178b210a3de68086f605396f7a8e6cbc7a7144a6595515bc88a4cdf71d5"} Jul 24 17:40:57.232837 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:57.232802 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zcvr7" event={"ID":"afa66580-b192-471f-a0a1-2eadd6d85817","Type":"ContainerStarted","Data":"ac6bab58a1d14577c704377a6988a8c721bb17c8af81f298ca4be6e1bb8acda1"} Jul 24 17:40:57.235073 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:57.235047 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" event={"ID":"aa20ebee-6777-4053-82f0-6add3bb8b26e","Type":"ContainerStarted","Data":"8a9e6267201e3f10b9f01951664224b855b20f0ded7e5e427036b42add582897"} Jul 24 17:40:57.235155 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:57.235094 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" event={"ID":"aa20ebee-6777-4053-82f0-6add3bb8b26e","Type":"ContainerStarted","Data":"6b8af7af37698afcc42d159818717ec67ef4c027e9cd10ca3aba718f191cfa50"} Jul 24 17:40:57.235372 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:57.235327 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:40:57.263892 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:57.263690 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" podStartSLOduration=30.263674588 podStartE2EDuration="30.263674588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:57.261956155 +0000 UTC m=+52.874962016" watchObservedRunningTime="2026-07-24 17:40:57.263674588 +0000 UTC m=+52.876680453" Jul 24 17:40:59.244805 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:59.244769 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tmsv8" event={"ID":"21676a3c-bd55-4e4a-ab8e-4275108751f2","Type":"ContainerStarted","Data":"c92236e525810b94bd6f7b10cc7da3c90ec743bda6c4b0a8452a87091b845dd1"} Jul 24 17:40:59.245278 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:59.244828 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tmsv8" event={"ID":"21676a3c-bd55-4e4a-ab8e-4275108751f2","Type":"ContainerStarted","Data":"f35822edf37bad1a4f0e2a4de4230325820289bef654b8c4405977e004784b0c"} Jul 24 17:40:59.245278 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:59.244908 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tmsv8" Jul 24 17:40:59.263932 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:40:59.263885 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tmsv8" podStartSLOduration=17.376818508 podStartE2EDuration="19.263867244s" totalImagesPullingTime="1.887048736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:59.263552975 +0000 UTC m=+54.876558843" watchObservedRunningTime="2026-07-24 17:40:59.263867244 +0000 UTC m=+54.876873107" Jul 24 17:41:02.257981 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:02.257886 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zcvr7" event={"ID":"afa66580-b192-471f-a0a1-2eadd6d85817","Type":"ContainerStarted","Data":"47c556b67bcaef064924060e06466af4d92720859a661b8f9c7b2ca92211398e"} Jul 24 17:41:02.441671 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:02.441635 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:02.587791 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:02.587652 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:02.701447 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:02.701415 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:03.584737 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.584707 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69d8678b4b-sj7rh"] Jul 24 17:41:03.599550 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.599521 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb"] Jul 24 17:41:03.695259 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.695206 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:03.743066 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.743037 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf"] Jul 24 17:41:03.743234 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.743166 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" Jul 24 17:41:03.747242 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.747195 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 24 17:41:03.747374 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.747329 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 24 17:41:03.747650 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.747625 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 24 17:41:03.747954 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.747919 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 24 17:41:03.785692 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.785308 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb"] Jul 24 17:41:03.785692 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.785370 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf"] Jul 24 17:41:03.785692 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.785394 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-794b4ccc5-mtfft"] Jul 24 17:41:03.785980 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.785809 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.790655 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.790503 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 24 17:41:03.792367 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.792342 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 24 17:41:03.792487 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.792345 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 24 17:41:03.792487 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.792436 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 24 17:41:03.819110 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.819014 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-794b4ccc5-mtfft"] Jul 24 17:41:03.819230 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.819122 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:03.844742 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.844647 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:03.861768 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.861740 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a484ac30-0fca-4a85-913e-f92377ef1189-hub\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.861887 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.861782 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a484ac30-0fca-4a85-913e-f92377ef1189-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.861887 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.861825 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5bmn\" (UniqueName: \"kubernetes.io/projected/f6569a51-ae5d-474a-87ab-3cb5a811921c-kube-api-access-d5bmn\") pod \"klusterlet-addon-workmgr-79fdd86f74-74rdb\" (UID: \"f6569a51-ae5d-474a-87ab-3cb5a811921c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" Jul 24 17:41:03.861887 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.861880 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6569a51-ae5d-474a-87ab-3cb5a811921c-tmp\") pod \"klusterlet-addon-workmgr-79fdd86f74-74rdb\" (UID: \"f6569a51-ae5d-474a-87ab-3cb5a811921c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" Jul 24 17:41:03.862030 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.861906 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a484ac30-0fca-4a85-913e-f92377ef1189-ca\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.862030 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.862001 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a484ac30-0fca-4a85-913e-f92377ef1189-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.862030 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.862021 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f6569a51-ae5d-474a-87ab-3cb5a811921c-klusterlet-config\") pod \"klusterlet-addon-workmgr-79fdd86f74-74rdb\" (UID: \"f6569a51-ae5d-474a-87ab-3cb5a811921c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" Jul 24 17:41:03.862172 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.862049 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a484ac30-0fca-4a85-913e-f92377ef1189-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.862172 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.862074 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m27s5\" (UniqueName: \"kubernetes.io/projected/a484ac30-0fca-4a85-913e-f92377ef1189-kube-api-access-m27s5\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.928458 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.928426 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:03.962599 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.962573 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/00ae222f-d539-492e-bbbe-5973669634e9-registry-certificates\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:03.962731 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.962612 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a484ac30-0fca-4a85-913e-f92377ef1189-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.962731 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.962634 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00ae222f-d539-492e-bbbe-5973669634e9-bound-sa-token\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:03.962731 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.962652 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f6569a51-ae5d-474a-87ab-3cb5a811921c-klusterlet-config\") pod \"klusterlet-addon-workmgr-79fdd86f74-74rdb\" (UID: \"f6569a51-ae5d-474a-87ab-3cb5a811921c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" Jul 24 17:41:03.962731 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.962673 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a484ac30-0fca-4a85-913e-f92377ef1189-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.962731 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.962696 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m27s5\" (UniqueName: \"kubernetes.io/projected/a484ac30-0fca-4a85-913e-f92377ef1189-kube-api-access-m27s5\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.962731 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.962722 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a484ac30-0fca-4a85-913e-f92377ef1189-hub\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.963045 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.962746 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00ae222f-d539-492e-bbbe-5973669634e9-registry-tls\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:03.963045 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.962770 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a484ac30-0fca-4a85-913e-f92377ef1189-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.963045 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.962903 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/00ae222f-d539-492e-bbbe-5973669634e9-trusted-ca\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:03.963045 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.962933 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5bmn\" (UniqueName: \"kubernetes.io/projected/f6569a51-ae5d-474a-87ab-3cb5a811921c-kube-api-access-d5bmn\") pod \"klusterlet-addon-workmgr-79fdd86f74-74rdb\" (UID: \"f6569a51-ae5d-474a-87ab-3cb5a811921c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" Jul 24 17:41:03.963045 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.962975 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/00ae222f-d539-492e-bbbe-5973669634e9-installation-pull-secrets\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:03.963045 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.963031 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/00ae222f-d539-492e-bbbe-5973669634e9-image-registry-private-configuration\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:03.963332 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.963059 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/00ae222f-d539-492e-bbbe-5973669634e9-ca-trust-extracted\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:03.963332 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.963099 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6569a51-ae5d-474a-87ab-3cb5a811921c-tmp\") pod \"klusterlet-addon-workmgr-79fdd86f74-74rdb\" (UID: \"f6569a51-ae5d-474a-87ab-3cb5a811921c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" Jul 24 17:41:03.963332 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.963185 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a484ac30-0fca-4a85-913e-f92377ef1189-ca\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.963332 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.963229 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqq49\" (UniqueName: \"kubernetes.io/projected/00ae222f-d539-492e-bbbe-5973669634e9-kube-api-access-cqq49\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:03.963530 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.963401 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a484ac30-0fca-4a85-913e-f92377ef1189-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.963586 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.963548 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6569a51-ae5d-474a-87ab-3cb5a811921c-tmp\") pod \"klusterlet-addon-workmgr-79fdd86f74-74rdb\" (UID: \"f6569a51-ae5d-474a-87ab-3cb5a811921c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" Jul 24 17:41:03.965517 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.965492 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a484ac30-0fca-4a85-913e-f92377ef1189-hub\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.965654 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.965634 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a484ac30-0fca-4a85-913e-f92377ef1189-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.965840 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.965820 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f6569a51-ae5d-474a-87ab-3cb5a811921c-klusterlet-config\") pod \"klusterlet-addon-workmgr-79fdd86f74-74rdb\" (UID: \"f6569a51-ae5d-474a-87ab-3cb5a811921c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" Jul 24 17:41:03.965876 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.965828 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a484ac30-0fca-4a85-913e-f92377ef1189-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.965876 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.965858 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a484ac30-0fca-4a85-913e-f92377ef1189-ca\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:03.973147 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.973124 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5bmn\" (UniqueName: \"kubernetes.io/projected/f6569a51-ae5d-474a-87ab-3cb5a811921c-kube-api-access-d5bmn\") pod \"klusterlet-addon-workmgr-79fdd86f74-74rdb\" (UID: \"f6569a51-ae5d-474a-87ab-3cb5a811921c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" Jul 24 17:41:03.974982 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:03.974962 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m27s5\" (UniqueName: \"kubernetes.io/projected/a484ac30-0fca-4a85-913e-f92377ef1189-kube-api-access-m27s5\") pod \"cluster-proxy-proxy-agent-5db8d685b4-xbtzf\" (UID: \"a484ac30-0fca-4a85-913e-f92377ef1189\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:04.013678 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.013644 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:04.055568 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.055545 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" Jul 24 17:41:04.063750 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.063725 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/00ae222f-d539-492e-bbbe-5973669634e9-image-registry-private-configuration\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.063832 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.063753 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/00ae222f-d539-492e-bbbe-5973669634e9-ca-trust-extracted\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.063832 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.063789 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqq49\" (UniqueName: \"kubernetes.io/projected/00ae222f-d539-492e-bbbe-5973669634e9-kube-api-access-cqq49\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.063832 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.063817 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/00ae222f-d539-492e-bbbe-5973669634e9-registry-certificates\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.063964 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.063855 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00ae222f-d539-492e-bbbe-5973669634e9-bound-sa-token\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.063964 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.063897 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00ae222f-d539-492e-bbbe-5973669634e9-registry-tls\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.064083 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.064064 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/00ae222f-d539-492e-bbbe-5973669634e9-trusted-ca\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.064140 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.064102 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/00ae222f-d539-492e-bbbe-5973669634e9-installation-pull-secrets\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.064243 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.064222 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/00ae222f-d539-492e-bbbe-5973669634e9-ca-trust-extracted\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.064727 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.064697 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/00ae222f-d539-492e-bbbe-5973669634e9-registry-certificates\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.064994 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.064974 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/00ae222f-d539-492e-bbbe-5973669634e9-trusted-ca\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.066126 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.066101 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/00ae222f-d539-492e-bbbe-5973669634e9-image-registry-private-configuration\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.066126 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.066122 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/00ae222f-d539-492e-bbbe-5973669634e9-registry-tls\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.066378 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.066361 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/00ae222f-d539-492e-bbbe-5973669634e9-installation-pull-secrets\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.074152 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.074134 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00ae222f-d539-492e-bbbe-5973669634e9-bound-sa-token\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.077721 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.077703 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqq49\" (UniqueName: \"kubernetes.io/projected/00ae222f-d539-492e-bbbe-5973669634e9-kube-api-access-cqq49\") pod \"image-registry-794b4ccc5-mtfft\" (UID: \"00ae222f-d539-492e-bbbe-5973669634e9\") " pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.112391 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.112341 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" Jul 24 17:41:04.128650 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.128632 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:04.584638 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.584598 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:04.877500 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.877421 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:04.978600 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:04.978530 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:05.079543 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:05.079294 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:05.170381 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:05.170356 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dwvml" Jul 24 17:41:05.213546 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:05.213405 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf"] Jul 24 17:41:05.216182 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:41:05.216158 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda484ac30_0fca_4a85_913e_f92377ef1189.slice/crio-da7a1c56b818cc8cb6230b768fb3bff88a6a277b2dc10d3c3e09ae9a473736fe WatchSource:0}: Error finding container da7a1c56b818cc8cb6230b768fb3bff88a6a277b2dc10d3c3e09ae9a473736fe: Status 404 returned error can't find the container with id da7a1c56b818cc8cb6230b768fb3bff88a6a277b2dc10d3c3e09ae9a473736fe Jul 24 17:41:05.269437 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:05.269415 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" event={"ID":"a484ac30-0fca-4a85-913e-f92377ef1189","Type":"ContainerStarted","Data":"da7a1c56b818cc8cb6230b768fb3bff88a6a277b2dc10d3c3e09ae9a473736fe"} Jul 24 17:41:05.434407 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:05.434379 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb"] Jul 24 17:41:05.436602 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:41:05.436579 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6569a51_ae5d_474a_87ab_3cb5a811921c.slice/crio-574c23b78bd2c81d2174cca1b3e082ec7ef5aa6a03c92bfd406887602583b67d WatchSource:0}: Error finding container 574c23b78bd2c81d2174cca1b3e082ec7ef5aa6a03c92bfd406887602583b67d: Status 404 returned error can't find the container with id 574c23b78bd2c81d2174cca1b3e082ec7ef5aa6a03c92bfd406887602583b67d Jul 24 17:41:05.441076 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:05.441054 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-794b4ccc5-mtfft"] Jul 24 17:41:05.443806 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:41:05.443782 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00ae222f_d539_492e_bbbe_5973669634e9.slice/crio-44b4e077e5a521d155c6e86f3f56ceee5b5936509d8eb4ac3bae9d21297eb03d WatchSource:0}: Error finding container 44b4e077e5a521d155c6e86f3f56ceee5b5936509d8eb4ac3bae9d21297eb03d: Status 404 returned error can't find the container with id 44b4e077e5a521d155c6e86f3f56ceee5b5936509d8eb4ac3bae9d21297eb03d Jul 24 17:41:05.780097 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:05.779983 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:05.838069 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:05.838038 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:06.187475 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:06.187426 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:06.248875 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:06.248780 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:06.279186 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:06.279125 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" event={"ID":"f6569a51-ae5d-474a-87ab-3cb5a811921c","Type":"ContainerStarted","Data":"574c23b78bd2c81d2174cca1b3e082ec7ef5aa6a03c92bfd406887602583b67d"} Jul 24 17:41:06.283432 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:06.283364 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zcvr7" event={"ID":"afa66580-b192-471f-a0a1-2eadd6d85817","Type":"ContainerStarted","Data":"1d16df856a731c3cd9d531ecaca66a8009b91ce180a0934633644ec81c462985"} Jul 24 17:41:06.287506 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:06.287478 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" event={"ID":"00ae222f-d539-492e-bbbe-5973669634e9","Type":"ContainerStarted","Data":"284c9ebef8fff8d33a0bc1c3da0c3b2a4cbb05e0014716f34bda2a6321a06bf8"} Jul 24 17:41:06.287632 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:06.287524 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" event={"ID":"00ae222f-d539-492e-bbbe-5973669634e9","Type":"ContainerStarted","Data":"44b4e077e5a521d155c6e86f3f56ceee5b5936509d8eb4ac3bae9d21297eb03d"} Jul 24 17:41:06.287850 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:06.287762 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:06.308424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:06.308374 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" podStartSLOduration=3.3083555000000002 podStartE2EDuration="3.3083555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:06.307146258 +0000 UTC m=+61.920152126" watchObservedRunningTime="2026-07-24 17:41:06.3083555 +0000 UTC m=+61.921361368" Jul 24 17:41:08.494719 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:08.494638 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:08.595348 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:08.595296 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:09.251112 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:09.251078 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tmsv8" Jul 24 17:41:09.620329 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:09.620247 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:09.680855 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:09.680801 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:09.751883 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:09.751848 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:10.573583 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.573543 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:10.622205 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.622165 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:41:10.622687 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.622247 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs\") pod \"network-metrics-daemon-2fs78\" (UID: \"fa9f496c-49f7-4a62-a5a9-1bb903fc553d\") " pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:41:10.624883 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.624852 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 17:41:10.625717 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.625687 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 17:41:10.635119 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.635099 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa9f496c-49f7-4a62-a5a9-1bb903fc553d-metrics-certs\") pod \"network-metrics-daemon-2fs78\" (UID: \"fa9f496c-49f7-4a62-a5a9-1bb903fc553d\") " pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:41:10.635119 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.635116 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3c435cc-32ae-4849-a167-4cf7d91b6832-original-pull-secret\") pod \"global-pull-secret-syncer-zp4x2\" (UID: \"a3c435cc-32ae-4849-a167-4cf7d91b6832\") " pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:41:10.637551 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.637506 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:10.722997 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.722961 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gv8m\" (UniqueName: \"kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m\") pod \"network-check-target-4q8f4\" (UID: \"af9a2eb4-19e8-4a4c-a655-5fd0c7889235\") " pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:41:10.726226 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.726184 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 17:41:10.736274 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.736242 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 17:41:10.747096 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.747072 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gv8m\" (UniqueName: \"kubernetes.io/projected/af9a2eb4-19e8-4a4c-a655-5fd0c7889235-kube-api-access-9gv8m\") pod \"network-check-target-4q8f4\" (UID: \"af9a2eb4-19e8-4a4c-a655-5fd0c7889235\") " pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:41:10.747665 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.747637 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zp4x2" Jul 24 17:41:10.757463 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.757440 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-62mvz\"" Jul 24 17:41:10.764217 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.764199 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t8nmh\"" Jul 24 17:41:10.765213 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.765197 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fs78" Jul 24 17:41:10.772371 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:10.772352 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:41:12.834683 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:12.834606 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2fs78"] Jul 24 17:41:12.835811 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:12.835653 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:12.839330 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:41:12.839304 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa9f496c_49f7_4a62_a5a9_1bb903fc553d.slice/crio-20ebffd90d977765dbac3b66a6d24f5e99e677f7090f52a3283c668c50c7ab16 WatchSource:0}: Error finding container 20ebffd90d977765dbac3b66a6d24f5e99e677f7090f52a3283c668c50c7ab16: Status 404 returned error can't find the container with id 20ebffd90d977765dbac3b66a6d24f5e99e677f7090f52a3283c668c50c7ab16 Jul 24 17:41:13.065228 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.065197 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zp4x2"] Jul 24 17:41:13.066208 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.066182 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4q8f4"] Jul 24 17:41:13.068000 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:41:13.067973 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf9a2eb4_19e8_4a4c_a655_5fd0c7889235.slice/crio-8c1d1e2637ad445013335af4219fda4d984746565447dc056afaddeb830daa88 WatchSource:0}: Error finding container 8c1d1e2637ad445013335af4219fda4d984746565447dc056afaddeb830daa88: Status 404 returned error can't find the container with id 8c1d1e2637ad445013335af4219fda4d984746565447dc056afaddeb830daa88 Jul 24 17:41:13.068602 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:41:13.068582 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c435cc_32ae_4849_a167_4cf7d91b6832.slice/crio-6682c626b0b095ec65f2c505019c575b4bba5ea67736f1b000dde7ec1462e248 WatchSource:0}: Error finding container 6682c626b0b095ec65f2c505019c575b4bba5ea67736f1b000dde7ec1462e248: Status 404 returned error can't find the container with id 6682c626b0b095ec65f2c505019c575b4bba5ea67736f1b000dde7ec1462e248 Jul 24 17:41:13.317909 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.317877 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zp4x2" event={"ID":"a3c435cc-32ae-4849-a167-4cf7d91b6832","Type":"ContainerStarted","Data":"6682c626b0b095ec65f2c505019c575b4bba5ea67736f1b000dde7ec1462e248"} Jul 24 17:41:13.320093 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.320061 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" event={"ID":"a484ac30-0fca-4a85-913e-f92377ef1189","Type":"ContainerStarted","Data":"00f028f59df72574396745325dc12a7f984c7bff1b3a2ca8f08cdad0da714982"} Jul 24 17:41:13.320239 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.320106 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" event={"ID":"a484ac30-0fca-4a85-913e-f92377ef1189","Type":"ContainerStarted","Data":"adfae15dc0386918961f6173af36ed7a01662d21892f89ccb68f82cbce744498"} Jul 24 17:41:13.320239 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.320128 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" event={"ID":"a484ac30-0fca-4a85-913e-f92377ef1189","Type":"ContainerStarted","Data":"2810b8c09f3183591b2802896e9161657a282ee9b57d8cc27f159f887093b0d8"} Jul 24 17:41:13.321449 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.321424 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" event={"ID":"f6569a51-ae5d-474a-87ab-3cb5a811921c","Type":"ContainerStarted","Data":"eacb8cec125d24cd3898c036c371c10729b42613e78b991fe53801ebe0c38d75"} Jul 24 17:41:13.321588 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.321573 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" Jul 24 17:41:13.322721 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.322695 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2fs78" event={"ID":"fa9f496c-49f7-4a62-a5a9-1bb903fc553d","Type":"ContainerStarted","Data":"20ebffd90d977765dbac3b66a6d24f5e99e677f7090f52a3283c668c50c7ab16"} Jul 24 17:41:13.323469 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.323450 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" Jul 24 17:41:13.324975 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.324930 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zcvr7" event={"ID":"afa66580-b192-471f-a0a1-2eadd6d85817","Type":"ContainerStarted","Data":"25e1d47604edb011226bf53e048a6513c47fa4bfe5aea3aaf1c3ebc3541eafa1"} Jul 24 17:41:13.326103 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.326083 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4q8f4" event={"ID":"af9a2eb4-19e8-4a4c-a655-5fd0c7889235","Type":"ContainerStarted","Data":"8c1d1e2637ad445013335af4219fda4d984746565447dc056afaddeb830daa88"} Jul 24 17:41:13.340257 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.340212 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5db8d685b4-xbtzf" podStartSLOduration=2.869858333 podStartE2EDuration="10.340199683s" totalImagesPullingTime="7.47034135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:13.338278441 +0000 UTC m=+68.951284328" watchObservedRunningTime="2026-07-24 17:41:13.340199683 +0000 UTC m=+68.953205562" Jul 24 17:41:13.362192 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.362148 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zcvr7" podStartSLOduration=17.123184282 podStartE2EDuration="33.362135663s" totalImagesPullingTime="16.238951381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:40:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:13.360766192 +0000 UTC m=+68.973772057" watchObservedRunningTime="2026-07-24 17:41:13.362135663 +0000 UTC m=+68.975141536" Jul 24 17:41:13.375786 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.375746 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79fdd86f74-74rdb" podStartSLOduration=3.126008039 podStartE2EDuration="10.37573298s" totalImagesPullingTime="7.249724941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:13.375147966 +0000 UTC m=+68.988153833" watchObservedRunningTime="2026-07-24 17:41:13.37573298 +0000 UTC m=+68.988738844" Jul 24 17:41:13.591659 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.591581 2561 patch_prober.go:28] interesting pod/image-registry-69d8678b4b-sj7rh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 24 17:41:13.591659 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:13.591641 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" podUID="aa20ebee-6777-4053-82f0-6add3bb8b26e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 17:41:14.333678 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:14.333111 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2fs78" event={"ID":"fa9f496c-49f7-4a62-a5a9-1bb903fc553d","Type":"ContainerStarted","Data":"da2d6ec54dee22405247dc56a443df1b62fdda4afb3cb3ebe9b4fbf3e36c5437"} Jul 24 17:41:14.333678 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:14.333220 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2fs78" event={"ID":"fa9f496c-49f7-4a62-a5a9-1bb903fc553d","Type":"ContainerStarted","Data":"e1fb19bcefdf82b1ae2447e8df0e2ca7a447a90eb20fd192cf7a11d485a7ef34"} Jul 24 17:41:14.349498 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:14.349449 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2fs78" podStartSLOduration=68.178835516 podStartE2EDuration="1m9.349432892s" totalImagesPullingTime="1.170597376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:14.348905894 +0000 UTC m=+69.961911760" watchObservedRunningTime="2026-07-24 17:41:14.349432892 +0000 UTC m=+69.962438757" Jul 24 17:41:14.454723 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:14.454678 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:14.555461 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:14.555422 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:14.775683 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:14.775644 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:14.998929 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:14.998133 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:15.055587 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.053340 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vfw2t"] Jul 24 17:41:15.055897 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.055878 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.066558 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.066532 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 17:41:15.066692 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.066581 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 17:41:15.066692 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.066668 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sft4t\"" Jul 24 17:41:15.066692 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.066580 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 17:41:15.066933 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.066898 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 17:41:15.067035 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.067024 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 17:41:15.067725 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.067704 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 17:41:15.100874 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.100843 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:15.167009 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.166959 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25543d3e-6225-4815-92aa-237ee763a248-root\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.167159 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.167030 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-wtmp\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.167159 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.167092 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-accelerators-collector-config\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.167159 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.167148 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25543d3e-6225-4815-92aa-237ee763a248-sys\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.167320 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.167202 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-tls\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.167320 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.167240 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25543d3e-6225-4815-92aa-237ee763a248-metrics-client-ca\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.167320 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.167268 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.167320 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.167299 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-textfile\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.167511 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.167323 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jfs5\" (UniqueName: \"kubernetes.io/projected/25543d3e-6225-4815-92aa-237ee763a248-kube-api-access-8jfs5\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.171895 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.171869 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:15.269592 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.268657 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25543d3e-6225-4815-92aa-237ee763a248-root\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.269752 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.269631 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-wtmp\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.269752 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.269672 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-accelerators-collector-config\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.269752 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.269714 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25543d3e-6225-4815-92aa-237ee763a248-sys\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.269752 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.269736 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-tls\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.270102 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.269759 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25543d3e-6225-4815-92aa-237ee763a248-metrics-client-ca\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.270102 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.269785 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.270102 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.269816 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-textfile\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.270102 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.269838 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jfs5\" (UniqueName: \"kubernetes.io/projected/25543d3e-6225-4815-92aa-237ee763a248-kube-api-access-8jfs5\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.270288 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.269552 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25543d3e-6225-4815-92aa-237ee763a248-root\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.270377 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.270361 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-wtmp\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.271085 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.270968 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-accelerators-collector-config\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.271085 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.271022 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25543d3e-6225-4815-92aa-237ee763a248-sys\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.271250 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:41:15.271093 2561 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 24 17:41:15.271250 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:41:15.271165 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-tls podName:25543d3e-6225-4815-92aa-237ee763a248 nodeName:}" failed. No retries permitted until 2026-07-24 17:41:15.771143837 +0000 UTC m=+71.384149684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-tls") pod "node-exporter-vfw2t" (UID: "25543d3e-6225-4815-92aa-237ee763a248") : secret "node-exporter-tls" not found Jul 24 17:41:15.271509 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.271474 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-textfile\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.271752 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.271725 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25543d3e-6225-4815-92aa-237ee763a248-metrics-client-ca\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.280403 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.280351 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:15.283524 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.283500 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.294454 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.294427 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jfs5\" (UniqueName: \"kubernetes.io/projected/25543d3e-6225-4815-92aa-237ee763a248-kube-api-access-8jfs5\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.377519 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.377438 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:15.476523 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.476486 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:15.528772 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.528731 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:15.646679 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.646603 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:15.774665 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.774629 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-tls\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.777388 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.777356 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25543d3e-6225-4815-92aa-237ee763a248-node-exporter-tls\") pod \"node-exporter-vfw2t\" (UID: \"25543d3e-6225-4815-92aa-237ee763a248\") " pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.981803 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.981760 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vfw2t" Jul 24 17:41:15.986917 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:15.986888 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:16.078582 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:16.078546 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:16.205932 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:16.205886 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:16.301816 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:16.301718 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:17.826748 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:17.826691 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:17.938960 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:17.938427 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:18.028203 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:18.028169 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:18.074847 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:41:18.074822 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25543d3e_6225_4815_92aa_237ee763a248.slice/crio-e27e787e4963cd21028302dc889b8dbc50b7d390de9d518cd978d30009ac0b7b WatchSource:0}: Error finding container e27e787e4963cd21028302dc889b8dbc50b7d390de9d518cd978d30009ac0b7b: Status 404 returned error can't find the container with id e27e787e4963cd21028302dc889b8dbc50b7d390de9d518cd978d30009ac0b7b Jul 24 17:41:18.127509 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:18.127478 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:18.229277 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:18.229242 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:18.350684 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:18.350594 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4q8f4" event={"ID":"af9a2eb4-19e8-4a4c-a655-5fd0c7889235","Type":"ContainerStarted","Data":"e77792ae9ea49e50cea176c172aad8a8a116246ff4564cb432b776221fa25192"} Jul 24 17:41:18.350846 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:18.350728 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:41:18.351806 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:18.351782 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vfw2t" event={"ID":"25543d3e-6225-4815-92aa-237ee763a248","Type":"ContainerStarted","Data":"e27e787e4963cd21028302dc889b8dbc50b7d390de9d518cd978d30009ac0b7b"} Jul 24 17:41:18.353106 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:18.353085 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zp4x2" event={"ID":"a3c435cc-32ae-4849-a167-4cf7d91b6832","Type":"ContainerStarted","Data":"c8c8dc2bff0d4d4689293f7f44a8bd3aaf9739308c9af6591d1b9e5fd43e8d18"} Jul 24 17:41:18.365733 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:18.365691 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4q8f4" podStartSLOduration=68.35959927 podStartE2EDuration="1m13.365677293s" totalImagesPullingTime="5.006078023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:18.364956674 +0000 UTC m=+73.977962533" watchObservedRunningTime="2026-07-24 17:41:18.365677293 +0000 UTC m=+73.978683197" Jul 24 17:41:18.380438 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:18.380396 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zp4x2" podStartSLOduration=68.36289755 podStartE2EDuration="1m13.380381889s" totalImagesPullingTime="5.017484339s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:18.378734986 +0000 UTC m=+73.991740852" watchObservedRunningTime="2026-07-24 17:41:18.380381889 +0000 UTC m=+73.993387756" Jul 24 17:41:19.806574 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:19.806493 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:19.962976 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:19.962924 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:20.361181 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:20.361146 2561 generic.go:356] "Generic (PLEG): container finished" podID="25543d3e-6225-4815-92aa-237ee763a248" containerID="6a656346c1a3b407b2513dace687f5bca690560294e1ed64e6e56b9a9d660d3f" exitCode=0 Jul 24 17:41:20.361349 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:20.361185 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vfw2t" event={"ID":"25543d3e-6225-4815-92aa-237ee763a248","Type":"ContainerDied","Data":"6a656346c1a3b407b2513dace687f5bca690560294e1ed64e6e56b9a9d660d3f"} Jul 24 17:41:21.231105 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:21.231074 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:21.366899 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:21.366854 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vfw2t" event={"ID":"25543d3e-6225-4815-92aa-237ee763a248","Type":"ContainerStarted","Data":"fcdd607871060ca6fa550c69b70ef108b752b45935df0d4c25d8ab52bacaa00f"} Jul 24 17:41:21.367073 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:21.366911 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vfw2t" event={"ID":"25543d3e-6225-4815-92aa-237ee763a248","Type":"ContainerStarted","Data":"227dd76b69c680721ce728fd1f84edba447b09783a6da764780cbe9585950212"} Jul 24 17:41:21.389193 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:21.389145 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vfw2t" podStartSLOduration=4.935280843 podStartE2EDuration="6.389131147s" totalImagesPullingTime="1.3905881s" totalInitContainerRuntime="63.262204ms" isStatefulPod=true podCreationTimestamp="2026-07-24 17:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:21.388169428 +0000 UTC m=+77.001175294" watchObservedRunningTime="2026-07-24 17:41:21.389131147 +0000 UTC m=+77.002137013" Jul 24 17:41:21.714735 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:21.714697 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:21.792984 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:21.792932 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:23.589620 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:23.589593 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:41:23.777385 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:23.777349 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:23.840531 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:23.840442 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:25.518996 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:25.518958 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:25.585958 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:25.585897 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:27.297411 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:27.297300 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-794b4ccc5-mtfft" Jul 24 17:41:27.424651 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:27.424616 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:27.482510 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:27.482481 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:27.669422 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:27.669341 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:27.769961 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:27.769904 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:28.605357 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.605319 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" podUID="aa20ebee-6777-4053-82f0-6add3bb8b26e" containerName="registry" containerID="cri-o://8a9e6267201e3f10b9f01951664224b855b20f0ded7e5e427036b42add582897" gracePeriod=30 Jul 24 17:41:28.849019 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.848994 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:41:28.974189 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.974163 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aa20ebee-6777-4053-82f0-6add3bb8b26e-trusted-ca\") pod \"aa20ebee-6777-4053-82f0-6add3bb8b26e\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " Jul 24 17:41:28.974372 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.974201 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aa20ebee-6777-4053-82f0-6add3bb8b26e-installation-pull-secrets\") pod \"aa20ebee-6777-4053-82f0-6add3bb8b26e\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " Jul 24 17:41:28.974372 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.974237 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aa20ebee-6777-4053-82f0-6add3bb8b26e-image-registry-private-configuration\") pod \"aa20ebee-6777-4053-82f0-6add3bb8b26e\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " Jul 24 17:41:28.974372 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.974262 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-bound-sa-token\") pod \"aa20ebee-6777-4053-82f0-6add3bb8b26e\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " Jul 24 17:41:28.974540 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.974376 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-certificates\") pod \"aa20ebee-6777-4053-82f0-6add3bb8b26e\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " Jul 24 17:41:28.974540 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.974437 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aa20ebee-6777-4053-82f0-6add3bb8b26e-ca-trust-extracted\") pod \"aa20ebee-6777-4053-82f0-6add3bb8b26e\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " Jul 24 17:41:28.974540 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.974461 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls\") pod \"aa20ebee-6777-4053-82f0-6add3bb8b26e\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " Jul 24 17:41:28.974540 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.974490 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzkcb\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-kube-api-access-gzkcb\") pod \"aa20ebee-6777-4053-82f0-6add3bb8b26e\" (UID: \"aa20ebee-6777-4053-82f0-6add3bb8b26e\") " Jul 24 17:41:28.974730 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.974678 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa20ebee-6777-4053-82f0-6add3bb8b26e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "aa20ebee-6777-4053-82f0-6add3bb8b26e" (UID: "aa20ebee-6777-4053-82f0-6add3bb8b26e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:41:28.974775 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.974749 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "aa20ebee-6777-4053-82f0-6add3bb8b26e" (UID: "aa20ebee-6777-4053-82f0-6add3bb8b26e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:41:28.977180 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.977149 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "aa20ebee-6777-4053-82f0-6add3bb8b26e" (UID: "aa20ebee-6777-4053-82f0-6add3bb8b26e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:41:28.977327 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.977223 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa20ebee-6777-4053-82f0-6add3bb8b26e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "aa20ebee-6777-4053-82f0-6add3bb8b26e" (UID: "aa20ebee-6777-4053-82f0-6add3bb8b26e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:41:28.977327 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.977303 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-kube-api-access-gzkcb" (OuterVolumeSpecName: "kube-api-access-gzkcb") pod "aa20ebee-6777-4053-82f0-6add3bb8b26e" (UID: "aa20ebee-6777-4053-82f0-6add3bb8b26e"). InnerVolumeSpecName "kube-api-access-gzkcb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:41:28.977327 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.977310 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa20ebee-6777-4053-82f0-6add3bb8b26e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "aa20ebee-6777-4053-82f0-6add3bb8b26e" (UID: "aa20ebee-6777-4053-82f0-6add3bb8b26e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:41:28.977449 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.977346 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "aa20ebee-6777-4053-82f0-6add3bb8b26e" (UID: "aa20ebee-6777-4053-82f0-6add3bb8b26e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:41:28.983825 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:28.983800 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa20ebee-6777-4053-82f0-6add3bb8b26e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "aa20ebee-6777-4053-82f0-6add3bb8b26e" (UID: "aa20ebee-6777-4053-82f0-6add3bb8b26e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:41:29.075816 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.075775 2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aa20ebee-6777-4053-82f0-6add3bb8b26e-image-registry-private-configuration\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 17:41:29.075816 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.075811 2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-bound-sa-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 17:41:29.076046 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.075826 2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-certificates\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 17:41:29.076046 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.075840 2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aa20ebee-6777-4053-82f0-6add3bb8b26e-ca-trust-extracted\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 17:41:29.076046 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.075852 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-registry-tls\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 17:41:29.076046 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.075862 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzkcb\" (UniqueName: \"kubernetes.io/projected/aa20ebee-6777-4053-82f0-6add3bb8b26e-kube-api-access-gzkcb\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 17:41:29.076046 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.075872 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aa20ebee-6777-4053-82f0-6add3bb8b26e-trusted-ca\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 17:41:29.076046 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.075883 2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aa20ebee-6777-4053-82f0-6add3bb8b26e-installation-pull-secrets\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 17:41:29.396758 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.396674 2561 generic.go:356] "Generic (PLEG): container finished" podID="aa20ebee-6777-4053-82f0-6add3bb8b26e" containerID="8a9e6267201e3f10b9f01951664224b855b20f0ded7e5e427036b42add582897" exitCode=0 Jul 24 17:41:29.396758 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.396738 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" Jul 24 17:41:29.396988 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.396738 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" event={"ID":"aa20ebee-6777-4053-82f0-6add3bb8b26e","Type":"ContainerDied","Data":"8a9e6267201e3f10b9f01951664224b855b20f0ded7e5e427036b42add582897"} Jul 24 17:41:29.396988 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.396849 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d8678b4b-sj7rh" event={"ID":"aa20ebee-6777-4053-82f0-6add3bb8b26e","Type":"ContainerDied","Data":"6b8af7af37698afcc42d159818717ec67ef4c027e9cd10ca3aba718f191cfa50"} Jul 24 17:41:29.396988 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.396879 2561 scope.go:117] "RemoveContainer" containerID="8a9e6267201e3f10b9f01951664224b855b20f0ded7e5e427036b42add582897" Jul 24 17:41:29.405256 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.405239 2561 scope.go:117] "RemoveContainer" containerID="8a9e6267201e3f10b9f01951664224b855b20f0ded7e5e427036b42add582897" Jul 24 17:41:29.405491 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:41:29.405473 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a9e6267201e3f10b9f01951664224b855b20f0ded7e5e427036b42add582897\": container with ID starting with 8a9e6267201e3f10b9f01951664224b855b20f0ded7e5e427036b42add582897 not found: ID does not exist" containerID="8a9e6267201e3f10b9f01951664224b855b20f0ded7e5e427036b42add582897" Jul 24 17:41:29.405537 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.405499 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a9e6267201e3f10b9f01951664224b855b20f0ded7e5e427036b42add582897"} err="failed to get container status \"8a9e6267201e3f10b9f01951664224b855b20f0ded7e5e427036b42add582897\": rpc error: code = NotFound desc = could not find container \"8a9e6267201e3f10b9f01951664224b855b20f0ded7e5e427036b42add582897\": container with ID starting with 8a9e6267201e3f10b9f01951664224b855b20f0ded7e5e427036b42add582897 not found: ID does not exist" Jul 24 17:41:29.420288 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.420261 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69d8678b4b-sj7rh"] Jul 24 17:41:29.434631 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:29.434609 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69d8678b4b-sj7rh"] Jul 24 17:41:30.939144 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:30.939111 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa20ebee-6777-4053-82f0-6add3bb8b26e" path="/var/lib/kubelet/pods/aa20ebee-6777-4053-82f0-6add3bb8b26e/volumes" Jul 24 17:41:31.137640 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:31.137603 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:31.238440 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:31.238370 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:34.325119 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:34.325075 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:34.422314 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:34.422282 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:34.523599 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:34.523568 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:34.622156 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:34.622078 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:34.721024 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:34.720987 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:34.801383 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:34.801342 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:35.431660 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:35.431626 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:35.813494 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:35.813406 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:35.914213 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:35.914171 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:36.013813 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:36.013776 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:36.115399 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:36.115318 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:36.216324 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:36.216291 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:36.317531 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:36.317489 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:36.415689 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:36.415613 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:38.503010 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:38.502973 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:38.605247 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:38.605209 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:40.209203 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:40.209166 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:40.306491 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:40.306454 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:48.822753 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:48.822715 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:48.876228 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:48.876177 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:49.359119 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:49.359087 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4q8f4" Jul 24 17:41:53.033279 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:53.033247 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:53.095596 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:53.095568 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:53.521604 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:53.521564 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:53.599479 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:53.599442 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:55.238167 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:55.238140 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:55.338976 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:55.338926 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:56.785626 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:56.785556 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:56.889383 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:56.889338 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:59.024616 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:59.024579 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:59.081509 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:59.081474 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:41:59.206838 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:41:59.206804 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:42:06.430750 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:42:06.430711 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:42:06.488537 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:42:06.488494 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:42:08.496851 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:42:08.496815 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:42:08.597403 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:42:08.597364 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:42:34.235389 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:42:34.235351 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:42:34.636398 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:42:34.636309 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:42:43.780877 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:42:43.780838 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:42:43.883528 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:42:43.883488 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:42:43.945845 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:42:43.945807 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:42:44.468218 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:42:44.468179 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:42:44.570533 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:42:44.570498 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:42:54.658500 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:42:54.658460 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:42:54.757218 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:42:54.757177 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:42:54.831874 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:42:54.831839 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:43:25.336909 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:43:25.336804 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:43:25.446327 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:43:25.446284 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:43:49.506884 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:43:49.506839 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:43:49.581794 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:43:49.581760 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:44:01.544523 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:44:01.544489 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:44:01.596600 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:44:01.596569 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:44:08.506316 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:44:08.506263 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:44:08.607856 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:44:08.607819 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:44:29.151040 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:44:29.150933 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:44:34.426250 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:44:34.426217 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:44:34.478301 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:44:34.478257 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:44:45.081437 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:44:45.081408 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:44:45.182785 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:44:45.182754 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:44:45.240868 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:44:45.240836 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:44:49.444384 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:44:49.444342 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:44:49.543103 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:44:49.543064 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:04.777772 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:04.777738 2561 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 17:45:16.300351 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:16.300300 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:17.773718 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:17.773681 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:20.293046 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:20.292993 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:20.361925 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:20.361889 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:21.495740 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:21.495696 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:21.597217 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:21.597178 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:21.886595 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:21.886485 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:21.984579 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:21.984544 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:23.609414 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:23.609378 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:24.209382 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:24.209345 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:26.956469 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:26.956431 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:27.056788 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:27.056730 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:28.780174 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:28.780130 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:28.844278 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:28.844239 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:32.503988 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:32.503897 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:32.606146 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:32.606111 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:33.016508 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:33.016475 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:38.916983 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:38.916949 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:39.022457 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:39.022417 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:40.885102 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:40.885063 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:53.909396 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:53.909354 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:54.008534 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.008499 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:54.522197 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.522155 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:54.722336 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.722304 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:54.829162 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.829073 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:54.844603 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.844570 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh"] Jul 24 17:45:54.844880 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.844864 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa20ebee-6777-4053-82f0-6add3bb8b26e" containerName="registry" Jul 24 17:45:54.844983 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.844884 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa20ebee-6777-4053-82f0-6add3bb8b26e" containerName="registry" Jul 24 17:45:54.844983 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.844965 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa20ebee-6777-4053-82f0-6add3bb8b26e" containerName="registry" Jul 24 17:45:54.847865 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.847846 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:54.851314 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.851271 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ng8ql\"" Jul 24 17:45:54.851484 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.851323 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 17:45:54.851484 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.851361 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 17:45:54.851630 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.851615 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 17:45:54.851984 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.851970 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 17:45:54.875635 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.875603 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh"] Jul 24 17:45:54.952470 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.952426 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4568\" (UniqueName: \"kubernetes.io/projected/3bb64cee-06b6-47c6-b80b-72f990f7b77e-kube-api-access-q4568\") pod \"opendatahub-operator-controller-manager-67f579866-4zsfh\" (UID: \"3bb64cee-06b6-47c6-b80b-72f990f7b77e\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:54.952470 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.952477 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bb64cee-06b6-47c6-b80b-72f990f7b77e-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f579866-4zsfh\" (UID: \"3bb64cee-06b6-47c6-b80b-72f990f7b77e\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:54.952900 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.952508 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3bb64cee-06b6-47c6-b80b-72f990f7b77e-manifests\") pod \"opendatahub-operator-controller-manager-67f579866-4zsfh\" (UID: \"3bb64cee-06b6-47c6-b80b-72f990f7b77e\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:54.952900 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.952555 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bb64cee-06b6-47c6-b80b-72f990f7b77e-tmp\") pod \"opendatahub-operator-controller-manager-67f579866-4zsfh\" (UID: \"3bb64cee-06b6-47c6-b80b-72f990f7b77e\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:54.952900 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:54.952586 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3bb64cee-06b6-47c6-b80b-72f990f7b77e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f579866-4zsfh\" (UID: \"3bb64cee-06b6-47c6-b80b-72f990f7b77e\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:55.010531 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.010497 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:55.053479 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.053441 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4568\" (UniqueName: \"kubernetes.io/projected/3bb64cee-06b6-47c6-b80b-72f990f7b77e-kube-api-access-q4568\") pod \"opendatahub-operator-controller-manager-67f579866-4zsfh\" (UID: \"3bb64cee-06b6-47c6-b80b-72f990f7b77e\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:55.053479 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.053485 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bb64cee-06b6-47c6-b80b-72f990f7b77e-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f579866-4zsfh\" (UID: \"3bb64cee-06b6-47c6-b80b-72f990f7b77e\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:55.053700 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.053523 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3bb64cee-06b6-47c6-b80b-72f990f7b77e-manifests\") pod \"opendatahub-operator-controller-manager-67f579866-4zsfh\" (UID: \"3bb64cee-06b6-47c6-b80b-72f990f7b77e\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:55.053700 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.053549 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bb64cee-06b6-47c6-b80b-72f990f7b77e-tmp\") pod \"opendatahub-operator-controller-manager-67f579866-4zsfh\" (UID: \"3bb64cee-06b6-47c6-b80b-72f990f7b77e\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:55.053700 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.053583 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3bb64cee-06b6-47c6-b80b-72f990f7b77e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f579866-4zsfh\" (UID: \"3bb64cee-06b6-47c6-b80b-72f990f7b77e\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:55.054069 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.054047 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3bb64cee-06b6-47c6-b80b-72f990f7b77e-manifests\") pod \"opendatahub-operator-controller-manager-67f579866-4zsfh\" (UID: \"3bb64cee-06b6-47c6-b80b-72f990f7b77e\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:55.055792 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.055767 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bb64cee-06b6-47c6-b80b-72f990f7b77e-tmp\") pod \"opendatahub-operator-controller-manager-67f579866-4zsfh\" (UID: \"3bb64cee-06b6-47c6-b80b-72f990f7b77e\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:55.055921 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.055910 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bb64cee-06b6-47c6-b80b-72f990f7b77e-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f579866-4zsfh\" (UID: \"3bb64cee-06b6-47c6-b80b-72f990f7b77e\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:55.056085 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.056064 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3bb64cee-06b6-47c6-b80b-72f990f7b77e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f579866-4zsfh\" (UID: \"3bb64cee-06b6-47c6-b80b-72f990f7b77e\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:55.064202 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.064177 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4568\" (UniqueName: \"kubernetes.io/projected/3bb64cee-06b6-47c6-b80b-72f990f7b77e-kube-api-access-q4568\") pod \"opendatahub-operator-controller-manager-67f579866-4zsfh\" (UID: \"3bb64cee-06b6-47c6-b80b-72f990f7b77e\") " pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:55.125285 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.125191 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:55.158304 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.158265 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:45:55.210412 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.210354 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:55.319855 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.319819 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh"] Jul 24 17:45:55.322813 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:45:55.322785 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bb64cee_06b6_47c6_b80b_72f990f7b77e.slice/crio-5215d7f03af21dee697b39d2e95b408630046e5f3ca2d11c373935f54a8431df WatchSource:0}: Error finding container 5215d7f03af21dee697b39d2e95b408630046e5f3ca2d11c373935f54a8431df: Status 404 returned error can't find the container with id 5215d7f03af21dee697b39d2e95b408630046e5f3ca2d11c373935f54a8431df Jul 24 17:45:55.324531 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.324514 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 17:45:55.340836 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.340803 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:55.451083 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.451046 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:55.553267 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:55.552066 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:56.266254 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:56.266215 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" event={"ID":"3bb64cee-06b6-47c6-b80b-72f990f7b77e","Type":"ContainerStarted","Data":"5215d7f03af21dee697b39d2e95b408630046e5f3ca2d11c373935f54a8431df"} Jul 24 17:45:57.216697 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:57.216595 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:57.316925 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:57.316893 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:45:59.279643 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:45:59.279602 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" event={"ID":"3bb64cee-06b6-47c6-b80b-72f990f7b77e","Type":"ContainerStarted","Data":"467b7875358da9ff5615547bbf9a386418c0cb9562d146f1abb9b4878eb84c42"} Jul 24 17:46:00.284533 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:00.284501 2561 generic.go:356] "Generic (PLEG): container finished" podID="3bb64cee-06b6-47c6-b80b-72f990f7b77e" containerID="467b7875358da9ff5615547bbf9a386418c0cb9562d146f1abb9b4878eb84c42" exitCode=0 Jul 24 17:46:00.284911 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:00.284550 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" event={"ID":"3bb64cee-06b6-47c6-b80b-72f990f7b77e","Type":"ContainerDied","Data":"467b7875358da9ff5615547bbf9a386418c0cb9562d146f1abb9b4878eb84c42"} Jul 24 17:46:01.289437 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:01.289397 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" event={"ID":"3bb64cee-06b6-47c6-b80b-72f990f7b77e","Type":"ContainerStarted","Data":"c36ae507ca2b67e0d60f49bced07edb6249058afeb487bbeba95831bd3865695"} Jul 24 17:46:01.289852 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:01.289532 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:46:01.312018 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:01.311965 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" podStartSLOduration=2.993752617 podStartE2EDuration="7.311933241s" totalImagesPullingTime="3.281375481s" totalInitContainerRuntime="1.036805143s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:45:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:01.310096373 +0000 UTC m=+356.923102264" watchObservedRunningTime="2026-07-24 17:46:01.311933241 +0000 UTC m=+356.924939107" Jul 24 17:46:06.111347 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:06.111305 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:06.220031 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:06.219993 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:11.926672 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:11.926633 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:12.295552 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:12.295518 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-67f579866-4zsfh" Jul 24 17:46:13.649496 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.649460 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt"] Jul 24 17:46:13.659735 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.659704 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:13.662226 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.662199 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 24 17:46:13.663368 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.663348 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 24 17:46:13.663466 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.663406 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hgdt2\"" Jul 24 17:46:13.663527 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.663465 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 24 17:46:13.663527 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.663491 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 24 17:46:13.663527 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.663500 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 24 17:46:13.669863 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.669837 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt"] Jul 24 17:46:13.792622 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.792582 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f0a75bb-350c-4cff-b6ae-b6266320aec3-cert\") pod \"lws-controller-manager-74c56c9c54-x99lt\" (UID: \"1f0a75bb-350c-4cff-b6ae-b6266320aec3\") " pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:13.792622 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.792631 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1f0a75bb-350c-4cff-b6ae-b6266320aec3-metrics-cert\") pod \"lws-controller-manager-74c56c9c54-x99lt\" (UID: \"1f0a75bb-350c-4cff-b6ae-b6266320aec3\") " pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:13.792831 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.792727 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1f0a75bb-350c-4cff-b6ae-b6266320aec3-manager-config\") pod \"lws-controller-manager-74c56c9c54-x99lt\" (UID: \"1f0a75bb-350c-4cff-b6ae-b6266320aec3\") " pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:13.792831 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.792769 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n26j\" (UniqueName: \"kubernetes.io/projected/1f0a75bb-350c-4cff-b6ae-b6266320aec3-kube-api-access-5n26j\") pod \"lws-controller-manager-74c56c9c54-x99lt\" (UID: \"1f0a75bb-350c-4cff-b6ae-b6266320aec3\") " pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:13.893109 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.893071 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1f0a75bb-350c-4cff-b6ae-b6266320aec3-metrics-cert\") pod \"lws-controller-manager-74c56c9c54-x99lt\" (UID: \"1f0a75bb-350c-4cff-b6ae-b6266320aec3\") " pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:13.893271 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.893128 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1f0a75bb-350c-4cff-b6ae-b6266320aec3-manager-config\") pod \"lws-controller-manager-74c56c9c54-x99lt\" (UID: \"1f0a75bb-350c-4cff-b6ae-b6266320aec3\") " pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:13.893271 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.893161 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n26j\" (UniqueName: \"kubernetes.io/projected/1f0a75bb-350c-4cff-b6ae-b6266320aec3-kube-api-access-5n26j\") pod \"lws-controller-manager-74c56c9c54-x99lt\" (UID: \"1f0a75bb-350c-4cff-b6ae-b6266320aec3\") " pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:13.893271 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.893201 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f0a75bb-350c-4cff-b6ae-b6266320aec3-cert\") pod \"lws-controller-manager-74c56c9c54-x99lt\" (UID: \"1f0a75bb-350c-4cff-b6ae-b6266320aec3\") " pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:13.893911 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.893886 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1f0a75bb-350c-4cff-b6ae-b6266320aec3-manager-config\") pod \"lws-controller-manager-74c56c9c54-x99lt\" (UID: \"1f0a75bb-350c-4cff-b6ae-b6266320aec3\") " pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:13.895596 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.895570 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1f0a75bb-350c-4cff-b6ae-b6266320aec3-metrics-cert\") pod \"lws-controller-manager-74c56c9c54-x99lt\" (UID: \"1f0a75bb-350c-4cff-b6ae-b6266320aec3\") " pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:13.895895 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.895873 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f0a75bb-350c-4cff-b6ae-b6266320aec3-cert\") pod \"lws-controller-manager-74c56c9c54-x99lt\" (UID: \"1f0a75bb-350c-4cff-b6ae-b6266320aec3\") " pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:13.917075 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.917006 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n26j\" (UniqueName: \"kubernetes.io/projected/1f0a75bb-350c-4cff-b6ae-b6266320aec3-kube-api-access-5n26j\") pod \"lws-controller-manager-74c56c9c54-x99lt\" (UID: \"1f0a75bb-350c-4cff-b6ae-b6266320aec3\") " pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:13.969064 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:13.969026 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:14.026021 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:14.025979 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:14.118092 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:14.118059 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt"] Jul 24 17:46:14.120899 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:46:14.120869 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f0a75bb_350c_4cff_b6ae_b6266320aec3.slice/crio-02c884efc8ede5841082f6495c559740ccd4723f80af5a9ce6aa293c12da9daa WatchSource:0}: Error finding container 02c884efc8ede5841082f6495c559740ccd4723f80af5a9ce6aa293c12da9daa: Status 404 returned error can't find the container with id 02c884efc8ede5841082f6495c559740ccd4723f80af5a9ce6aa293c12da9daa Jul 24 17:46:14.137654 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:14.137626 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:14.335043 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:14.335009 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" event={"ID":"1f0a75bb-350c-4cff-b6ae-b6266320aec3","Type":"ContainerStarted","Data":"02c884efc8ede5841082f6495c559740ccd4723f80af5a9ce6aa293c12da9daa"} Jul 24 17:46:14.629749 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:14.629665 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:14.849521 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:14.849488 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:14.944891 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:14.944856 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:15.045469 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:15.045436 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:15.233647 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:15.233547 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:15.334779 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:15.334718 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:15.691159 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:15.691124 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:15.987160 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:15.987078 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:16.086210 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:16.086174 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:17.121342 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:17.121305 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:17.348875 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:17.348836 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" event={"ID":"1f0a75bb-350c-4cff-b6ae-b6266320aec3","Type":"ContainerStarted","Data":"0b2910efe3e7085230b0b60c0d563d37a00c14491af3bce5210f63354fe43987"} Jul 24 17:46:17.349090 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:17.348992 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:17.386212 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:17.386098 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" podStartSLOduration=1.837721361 podStartE2EDuration="4.386081756s" totalImagesPullingTime="2.548360395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:17.384853435 +0000 UTC m=+372.997859301" watchObservedRunningTime="2026-07-24 17:46:17.386081756 +0000 UTC m=+372.999087621" Jul 24 17:46:21.376712 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:21.376679 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:21.461581 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:21.461538 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:22.026290 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.026255 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c"] Jul 24 17:46:22.035424 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.035389 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.038425 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.038399 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 17:46:22.038572 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.038441 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 17:46:22.038572 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.038441 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-522l2\"" Jul 24 17:46:22.038572 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.038532 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 17:46:22.043606 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.043294 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c"] Jul 24 17:46:22.063356 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.063321 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:22.163432 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.163398 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7197fed7-f249-49db-8de0-851291502d13-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.163615 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.163446 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7197fed7-f249-49db-8de0-851291502d13-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.163615 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.163474 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7197fed7-f249-49db-8de0-851291502d13-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.163615 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.163543 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7197fed7-f249-49db-8de0-851291502d13-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.163615 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.163583 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-497gv\" (UniqueName: \"kubernetes.io/projected/7197fed7-f249-49db-8de0-851291502d13-kube-api-access-497gv\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.163767 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.163640 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7197fed7-f249-49db-8de0-851291502d13-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.163767 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.163657 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7197fed7-f249-49db-8de0-851291502d13-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.163767 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.163681 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7197fed7-f249-49db-8de0-851291502d13-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.163767 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.163723 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7197fed7-f249-49db-8de0-851291502d13-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.233957 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.233908 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:22.264091 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.264051 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7197fed7-f249-49db-8de0-851291502d13-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.264091 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.264092 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7197fed7-f249-49db-8de0-851291502d13-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.264341 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.264147 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7197fed7-f249-49db-8de0-851291502d13-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.264341 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.264185 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-497gv\" (UniqueName: \"kubernetes.io/projected/7197fed7-f249-49db-8de0-851291502d13-kube-api-access-497gv\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.264341 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.264259 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7197fed7-f249-49db-8de0-851291502d13-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.264341 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.264289 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7197fed7-f249-49db-8de0-851291502d13-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.264341 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.264315 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7197fed7-f249-49db-8de0-851291502d13-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.264574 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.264345 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7197fed7-f249-49db-8de0-851291502d13-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.264574 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.264375 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7197fed7-f249-49db-8de0-851291502d13-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.264752 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.264704 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7197fed7-f249-49db-8de0-851291502d13-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.264866 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.264801 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7197fed7-f249-49db-8de0-851291502d13-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.264866 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.264831 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7197fed7-f249-49db-8de0-851291502d13-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.265012 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.264878 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7197fed7-f249-49db-8de0-851291502d13-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.265152 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.265131 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7197fed7-f249-49db-8de0-851291502d13-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.266627 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.266600 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7197fed7-f249-49db-8de0-851291502d13-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.266928 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.266908 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7197fed7-f249-49db-8de0-851291502d13-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.276506 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.276442 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7197fed7-f249-49db-8de0-851291502d13-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.276632 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.276612 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-497gv\" (UniqueName: \"kubernetes.io/projected/7197fed7-f249-49db-8de0-851291502d13-kube-api-access-497gv\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c\" (UID: \"7197fed7-f249-49db-8de0-851291502d13\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.347284 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.347249 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:22.365531 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.365479 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:22.489593 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:22.489559 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c"] Jul 24 17:46:22.492882 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:46:22.492857 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7197fed7_f249_49db_8de0_851291502d13.slice/crio-ddeffdfc815177295cd93a5040fe0ee04f8ccc06afecd6f607f2afb4afec2391 WatchSource:0}: Error finding container ddeffdfc815177295cd93a5040fe0ee04f8ccc06afecd6f607f2afb4afec2391: Status 404 returned error can't find the container with id ddeffdfc815177295cd93a5040fe0ee04f8ccc06afecd6f607f2afb4afec2391 Jul 24 17:46:23.374034 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:23.373986 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" event={"ID":"7197fed7-f249-49db-8de0-851291502d13","Type":"ContainerStarted","Data":"ddeffdfc815177295cd93a5040fe0ee04f8ccc06afecd6f607f2afb4afec2391"} Jul 24 17:46:23.940456 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:23.940413 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw"] Jul 24 17:46:23.943918 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:23.943896 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:23.947490 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:23.947457 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 24 17:46:23.947985 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:23.947960 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 24 17:46:23.947985 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:23.947975 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-54ddn\"" Jul 24 17:46:23.948467 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:23.948448 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 24 17:46:23.967431 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:23.967393 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw"] Jul 24 17:46:23.981452 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:23.981400 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:23.981452 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:23.981476 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:23.981727 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:23.981503 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:23.981727 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:23.981547 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:23.981727 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:23.981581 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgcjp\" (UniqueName: \"kubernetes.io/projected/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-kube-api-access-sgcjp\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:23.981727 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:23.981658 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:23.981727 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:23.981713 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:24.082549 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:24.082516 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:24.082749 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:24.082575 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:24.082749 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:24.082632 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:24.082749 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:24.082657 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:24.082749 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:24.082688 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:24.082749 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:24.082723 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgcjp\" (UniqueName: \"kubernetes.io/projected/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-kube-api-access-sgcjp\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:24.083048 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:24.082757 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:24.083859 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:24.083795 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:24.085360 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:24.085314 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:24.085460 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:24.085379 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:24.085520 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:24.085451 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:24.085575 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:24.085550 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:24.097374 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:24.097349 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgcjp\" (UniqueName: \"kubernetes.io/projected/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-kube-api-access-sgcjp\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:24.097483 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:24.097461 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ddbbb6d-d271-498f-bb27-2259d6b4e9b7-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-lxrvw\" (UID: \"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:24.256135 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:24.256054 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:25.203861 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:25.203810 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 17:46:25.204238 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:25.203895 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 17:46:25.204238 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:25.203924 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 17:46:25.348395 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:25.348359 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw"] Jul 24 17:46:25.352473 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:46:25.352432 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ddbbb6d_d271_498f_bb27_2259d6b4e9b7.slice/crio-ddd9679750164c6bc92a53b2c6863a93f6064dab63384f7b819fdf1431c31649 WatchSource:0}: Error finding container ddd9679750164c6bc92a53b2c6863a93f6064dab63384f7b819fdf1431c31649: Status 404 returned error can't find the container with id ddd9679750164c6bc92a53b2c6863a93f6064dab63384f7b819fdf1431c31649 Jul 24 17:46:25.384017 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:25.383974 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" event={"ID":"7197fed7-f249-49db-8de0-851291502d13","Type":"ContainerStarted","Data":"b0f851cee1753bf493ecc443b7498a4f0fa5fd1d11b1c638f3d899e1b782ea0d"} Jul 24 17:46:25.385505 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:25.385468 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" event={"ID":"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7","Type":"ContainerStarted","Data":"ddd9679750164c6bc92a53b2c6863a93f6064dab63384f7b819fdf1431c31649"} Jul 24 17:46:25.526721 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:25.526689 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:25.626153 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:25.626119 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:25.705392 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:25.705347 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:26.222657 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:26.222599 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:26.347835 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:26.347797 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:26.353068 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:26.353038 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:26.379219 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:26.379164 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" podStartSLOduration=1.670505366 podStartE2EDuration="4.379147264s" totalImagesPullingTime="2.708641898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:25.420631857 +0000 UTC m=+381.033637724" watchObservedRunningTime="2026-07-24 17:46:26.379147264 +0000 UTC m=+381.992153130" Jul 24 17:46:26.389504 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:26.389470 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:26.390501 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:26.390475 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c" Jul 24 17:46:26.818696 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:26.818657 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:26.923606 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:26.923537 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:27.120705 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:27.120606 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:28.004233 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:28.004194 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 17:46:28.004643 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:28.004277 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 17:46:28.354876 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:28.354786 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-74c56c9c54-x99lt" Jul 24 17:46:28.401159 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:28.401120 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" event={"ID":"9ddbbb6d-d271-498f-bb27-2259d6b4e9b7","Type":"ContainerStarted","Data":"89e8614ba110ba7c9963c0f0f4cf7da43c400e4ae8932839aed10b1b8e9d0abe"} Jul 24 17:46:28.401490 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:28.401463 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:28.421502 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:28.421446 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" podStartSLOduration=2.77255435 podStartE2EDuration="5.421426838s" totalImagesPullingTime="2.648872488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:28.420376832 +0000 UTC m=+384.033382699" watchObservedRunningTime="2026-07-24 17:46:28.421426838 +0000 UTC m=+384.034432705" Jul 24 17:46:29.407590 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:29.407557 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lxrvw" Jul 24 17:46:50.557415 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:50.557374 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:46:50.616189 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:46:50.616140 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:15.583114 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:15.583075 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:15.678703 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:15.678658 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:15.778721 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:15.778684 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:16.544047 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:16.544009 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:16.763501 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:16.763466 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:36.873117 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:36.873037 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:40.269916 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:40.269880 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:41.071272 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:41.071237 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:44.117726 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:44.117687 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:44.272488 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:44.272447 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:46.738058 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:46.738023 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:48.532234 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:48.532200 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:49.682954 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:49.682900 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:50.682764 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:50.682724 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:50.780710 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:50.780674 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:51.726341 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:51.726296 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:52.037153 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:52.037072 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:52.130055 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:52.130018 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:52.230390 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:52.230355 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:52.713796 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:52.713758 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:52.813837 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:52.813802 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:53.339645 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:53.339611 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:53.435834 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:53.435801 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:56.300153 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:56.300121 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:56.399581 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:56.399547 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:57.267668 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:57.267628 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:57.380110 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:57.380075 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:57.669601 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:57.669516 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:47:57.775736 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:47:57.775699 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:01.782111 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:01.782071 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:01.877976 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:01.877916 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:03.179654 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:03.179622 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:03.279488 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:03.279442 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:03.380379 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:03.380348 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:08.218905 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:08.218864 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:08.314503 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:08.314467 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:12.556920 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:12.556877 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:12.658369 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:12.658325 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:16.362573 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:16.362532 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:16.420089 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:16.420055 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:16.549200 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:16.549161 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:16.603985 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:16.603957 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:21.558503 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:21.558468 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:21.622963 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:21.622907 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:28.218192 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:28.218151 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:28.284954 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:28.284896 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:39.013080 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:39.013043 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:39.311732 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:39.311637 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:54.446040 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:54.446008 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:54.549784 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:54.549747 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:54.610622 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:54.610577 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:55.281578 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:55.281542 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:48:55.373957 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:48:55.373908 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:49:10.914509 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:10.914422 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:49:11.007086 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:11.007054 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:49:11.495534 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:11.495502 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:49:23.898013 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:23.897980 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:49:24.244402 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:24.244361 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:49:24.385760 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:24.385710 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-q6hvw"] Jul 24 17:49:24.388807 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:24.388789 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-q6hvw" Jul 24 17:49:24.391979 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:24.391956 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cnq2w\"" Jul 24 17:49:24.392086 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:24.391981 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 17:49:24.392086 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:24.391962 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 24 17:49:24.392086 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:24.391956 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 17:49:24.397144 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:24.397122 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-q6hvw"] Jul 24 17:49:24.434403 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:24.434376 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-q6hvw"] Jul 24 17:49:24.434650 ip-10-0-137-185 kubenswrapper[2561]: E0724 17:49:24.434628 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rnplr tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-rnplr tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-q6hvw" podUID="685c6b2d-b420-40ed-9c3f-f0c6a1c50296" Jul 24 17:49:24.484890 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:24.484861 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/685c6b2d-b420-40ed-9c3f-f0c6a1c50296-tls-cert\") pod \"authorino-76877994-q6hvw\" (UID: \"685c6b2d-b420-40ed-9c3f-f0c6a1c50296\") " pod="kuadrant-system/authorino-76877994-q6hvw" Jul 24 17:49:24.485053 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:24.484956 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnplr\" (UniqueName: \"kubernetes.io/projected/685c6b2d-b420-40ed-9c3f-f0c6a1c50296-kube-api-access-rnplr\") pod \"authorino-76877994-q6hvw\" (UID: \"685c6b2d-b420-40ed-9c3f-f0c6a1c50296\") " pod="kuadrant-system/authorino-76877994-q6hvw" Jul 24 17:49:24.585743 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:24.585667 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnplr\" (UniqueName: \"kubernetes.io/projected/685c6b2d-b420-40ed-9c3f-f0c6a1c50296-kube-api-access-rnplr\") pod \"authorino-76877994-q6hvw\" (UID: \"685c6b2d-b420-40ed-9c3f-f0c6a1c50296\") " pod="kuadrant-system/authorino-76877994-q6hvw" Jul 24 17:49:24.585743 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:24.585707 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/685c6b2d-b420-40ed-9c3f-f0c6a1c50296-tls-cert\") pod \"authorino-76877994-q6hvw\" (UID: \"685c6b2d-b420-40ed-9c3f-f0c6a1c50296\") " pod="kuadrant-system/authorino-76877994-q6hvw" Jul 24 17:49:24.588036 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:24.588008 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/685c6b2d-b420-40ed-9c3f-f0c6a1c50296-tls-cert\") pod \"authorino-76877994-q6hvw\" (UID: \"685c6b2d-b420-40ed-9c3f-f0c6a1c50296\") " pod="kuadrant-system/authorino-76877994-q6hvw" Jul 24 17:49:24.595462 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:24.595433 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnplr\" (UniqueName: \"kubernetes.io/projected/685c6b2d-b420-40ed-9c3f-f0c6a1c50296-kube-api-access-rnplr\") pod \"authorino-76877994-q6hvw\" (UID: \"685c6b2d-b420-40ed-9c3f-f0c6a1c50296\") " pod="kuadrant-system/authorino-76877994-q6hvw" Jul 24 17:49:25.034719 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:25.034686 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-q6hvw" Jul 24 17:49:25.041847 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:25.041821 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-q6hvw" Jul 24 17:49:25.088702 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:25.088659 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnplr\" (UniqueName: \"kubernetes.io/projected/685c6b2d-b420-40ed-9c3f-f0c6a1c50296-kube-api-access-rnplr\") pod \"685c6b2d-b420-40ed-9c3f-f0c6a1c50296\" (UID: \"685c6b2d-b420-40ed-9c3f-f0c6a1c50296\") " Jul 24 17:49:25.088702 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:25.088701 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/685c6b2d-b420-40ed-9c3f-f0c6a1c50296-tls-cert\") pod \"685c6b2d-b420-40ed-9c3f-f0c6a1c50296\" (UID: \"685c6b2d-b420-40ed-9c3f-f0c6a1c50296\") " Jul 24 17:49:25.090817 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:25.090778 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/685c6b2d-b420-40ed-9c3f-f0c6a1c50296-kube-api-access-rnplr" (OuterVolumeSpecName: "kube-api-access-rnplr") pod "685c6b2d-b420-40ed-9c3f-f0c6a1c50296" (UID: "685c6b2d-b420-40ed-9c3f-f0c6a1c50296"). InnerVolumeSpecName "kube-api-access-rnplr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:49:25.090921 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:25.090786 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/685c6b2d-b420-40ed-9c3f-f0c6a1c50296-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "685c6b2d-b420-40ed-9c3f-f0c6a1c50296" (UID: "685c6b2d-b420-40ed-9c3f-f0c6a1c50296"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:49:25.189738 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:25.189704 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnplr\" (UniqueName: \"kubernetes.io/projected/685c6b2d-b420-40ed-9c3f-f0c6a1c50296-kube-api-access-rnplr\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 17:49:25.189738 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:25.189731 2561 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/685c6b2d-b420-40ed-9c3f-f0c6a1c50296-tls-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 17:49:26.039151 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:26.039122 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-q6hvw" Jul 24 17:49:26.067986 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:26.067960 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-q6hvw"] Jul 24 17:49:26.071965 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:26.071930 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-q6hvw"] Jul 24 17:49:26.940452 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:26.940418 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="685c6b2d-b420-40ed-9c3f-f0c6a1c50296" path="/var/lib/kubelet/pods/685c6b2d-b420-40ed-9c3f-f0c6a1c50296/volumes" Jul 24 17:49:32.936085 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:32.936041 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:49:33.030674 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:33.030640 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:49:33.123744 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:33.123710 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:49:36.056400 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:36.056358 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:49:36.253391 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:36.253357 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:49:48.438838 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:48.438791 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:49:48.502992 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:48.502959 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:49:55.677960 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:49:55.677910 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:50:07.222157 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:07.222119 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:50:07.319337 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:07.319305 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:50:07.837836 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:07.837794 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:50:07.955580 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:07.955543 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:50:08.373187 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:08.373148 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:50:08.471931 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:08.471895 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:50:13.159617 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:13.159583 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:50:18.956296 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:18.956248 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:50:20.206786 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:20.206743 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:50:24.013764 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:24.013729 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:50:35.510402 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.510321 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq"] Jul 24 17:50:35.513405 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.513385 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.516970 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.516925 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 24 17:50:35.517073 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.516925 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 24 17:50:35.517073 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.516954 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k52rs\"" Jul 24 17:50:35.517267 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.517247 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 24 17:50:35.525896 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.525871 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq"] Jul 24 17:50:35.526305 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.526286 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2rjv\" (UniqueName: \"kubernetes.io/projected/33e860bf-aba3-43a3-a09e-dffcdb381294-kube-api-access-q2rjv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.526389 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.526337 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33e860bf-aba3-43a3-a09e-dffcdb381294-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.526389 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.526371 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33e860bf-aba3-43a3-a09e-dffcdb381294-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.526500 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.526397 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33e860bf-aba3-43a3-a09e-dffcdb381294-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.526500 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.526456 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33e860bf-aba3-43a3-a09e-dffcdb381294-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.526500 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.526492 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33e860bf-aba3-43a3-a09e-dffcdb381294-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.554108 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.554079 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:50:35.626989 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.626927 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33e860bf-aba3-43a3-a09e-dffcdb381294-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.627170 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.627007 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33e860bf-aba3-43a3-a09e-dffcdb381294-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.627170 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.627039 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33e860bf-aba3-43a3-a09e-dffcdb381294-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.627170 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.627094 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33e860bf-aba3-43a3-a09e-dffcdb381294-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.627170 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.627129 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33e860bf-aba3-43a3-a09e-dffcdb381294-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.627399 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.627190 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2rjv\" (UniqueName: \"kubernetes.io/projected/33e860bf-aba3-43a3-a09e-dffcdb381294-kube-api-access-q2rjv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.627454 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.627404 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33e860bf-aba3-43a3-a09e-dffcdb381294-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.627454 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.627421 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33e860bf-aba3-43a3-a09e-dffcdb381294-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.627602 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.627575 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33e860bf-aba3-43a3-a09e-dffcdb381294-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.629487 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.629456 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33e860bf-aba3-43a3-a09e-dffcdb381294-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.629676 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.629652 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33e860bf-aba3-43a3-a09e-dffcdb381294-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.638879 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.638850 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2rjv\" (UniqueName: \"kubernetes.io/projected/33e860bf-aba3-43a3-a09e-dffcdb381294-kube-api-access-q2rjv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mpxcq\" (UID: \"33e860bf-aba3-43a3-a09e-dffcdb381294\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.823191 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.823098 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:35.952717 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:35.952686 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq"] Jul 24 17:50:35.955154 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:50:35.955128 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e860bf_aba3_43a3_a09e_dffcdb381294.slice/crio-bf349735f4e915625cf23de89eee84f9b3af0a035402e0d8794872a01e70ab74 WatchSource:0}: Error finding container bf349735f4e915625cf23de89eee84f9b3af0a035402e0d8794872a01e70ab74: Status 404 returned error can't find the container with id bf349735f4e915625cf23de89eee84f9b3af0a035402e0d8794872a01e70ab74 Jul 24 17:50:36.282853 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:36.282806 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" event={"ID":"33e860bf-aba3-43a3-a09e-dffcdb381294","Type":"ContainerStarted","Data":"bf349735f4e915625cf23de89eee84f9b3af0a035402e0d8794872a01e70ab74"} Jul 24 17:50:37.288014 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:37.287981 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" event={"ID":"33e860bf-aba3-43a3-a09e-dffcdb381294","Type":"ContainerStarted","Data":"87dc9bbf399b8692d5547869cfb6d09ba85dcf0cf83682d026881d0694a4701d"} Jul 24 17:50:37.304796 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:37.304744 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" podStartSLOduration=1.365428582 podStartE2EDuration="2.304728452s" totalImagesPullingTime="939.29987ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:50:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:50:37.30385575 +0000 UTC m=+632.916861630" watchObservedRunningTime="2026-07-24 17:50:37.304728452 +0000 UTC m=+632.917734318" Jul 24 17:50:45.823960 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:45.823895 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:45.824395 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:45.823987 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:45.836330 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:45.836295 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:46.334682 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.334653 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mpxcq" Jul 24 17:50:46.425607 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.425581 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9"] Jul 24 17:50:46.428651 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.428635 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.430973 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.430948 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 24 17:50:46.447020 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.446996 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9"] Jul 24 17:50:46.483246 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.483211 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:50:46.508089 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.508056 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3089e9ab-532b-496f-b595-1a670a39a96f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.508089 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.508090 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3089e9ab-532b-496f-b595-1a670a39a96f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.508291 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.508191 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3089e9ab-532b-496f-b595-1a670a39a96f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.508291 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.508224 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65gst\" (UniqueName: \"kubernetes.io/projected/3089e9ab-532b-496f-b595-1a670a39a96f-kube-api-access-65gst\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.508291 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.508271 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3089e9ab-532b-496f-b595-1a670a39a96f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.508389 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.508302 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3089e9ab-532b-496f-b595-1a670a39a96f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.609038 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.608916 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3089e9ab-532b-496f-b595-1a670a39a96f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.609038 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.609014 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3089e9ab-532b-496f-b595-1a670a39a96f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.609038 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.609032 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3089e9ab-532b-496f-b595-1a670a39a96f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.609331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.609063 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3089e9ab-532b-496f-b595-1a670a39a96f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.609331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.609086 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65gst\" (UniqueName: \"kubernetes.io/projected/3089e9ab-532b-496f-b595-1a670a39a96f-kube-api-access-65gst\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.609331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.609115 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3089e9ab-532b-496f-b595-1a670a39a96f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.609331 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.609300 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3089e9ab-532b-496f-b595-1a670a39a96f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.609519 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.609371 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3089e9ab-532b-496f-b595-1a670a39a96f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.609519 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.609430 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3089e9ab-532b-496f-b595-1a670a39a96f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.611341 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.611316 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3089e9ab-532b-496f-b595-1a670a39a96f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.611548 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.611527 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3089e9ab-532b-496f-b595-1a670a39a96f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.619501 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.619466 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65gst\" (UniqueName: \"kubernetes.io/projected/3089e9ab-532b-496f-b595-1a670a39a96f-kube-api-access-65gst\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hczg9\" (UID: \"3089e9ab-532b-496f-b595-1a670a39a96f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.713448 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.713372 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:50:46.738430 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.738404 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:46.869387 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:46.869315 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9"] Jul 24 17:50:46.872555 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:50:46.872521 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3089e9ab_532b_496f_b595_1a670a39a96f.slice/crio-97f36afdb77576f9828e307dd4aa418fdfc93765a35afbc5f76681ee6df5d372 WatchSource:0}: Error finding container 97f36afdb77576f9828e307dd4aa418fdfc93765a35afbc5f76681ee6df5d372: Status 404 returned error can't find the container with id 97f36afdb77576f9828e307dd4aa418fdfc93765a35afbc5f76681ee6df5d372 Jul 24 17:50:47.328226 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:47.328188 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" event={"ID":"3089e9ab-532b-496f-b595-1a670a39a96f","Type":"ContainerStarted","Data":"4459d16935ae95fdc287d2d55071ddcfcb8cb9a520375c5f328847b7a1a3af96"} Jul 24 17:50:47.328404 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:47.328253 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" event={"ID":"3089e9ab-532b-496f-b595-1a670a39a96f","Type":"ContainerStarted","Data":"97f36afdb77576f9828e307dd4aa418fdfc93765a35afbc5f76681ee6df5d372"} Jul 24 17:50:47.363226 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:47.363166 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" podStartSLOduration=1.203378999 podStartE2EDuration="1.363143626s" totalImagesPullingTime="159.764627ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:50:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:50:47.361898282 +0000 UTC m=+642.974904144" watchObservedRunningTime="2026-07-24 17:50:47.363143626 +0000 UTC m=+642.976149493" Jul 24 17:50:56.739159 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:56.739120 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:56.739636 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:56.739227 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:56.751505 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:56.751480 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:57.378667 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:57.378635 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hczg9" Jul 24 17:50:58.402212 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:50:58.402178 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:51:20.764542 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:20.764508 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk"] Jul 24 17:51:20.767849 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:20.767833 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:20.771547 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:20.771526 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 24 17:51:20.810548 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:20.810521 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk"] Jul 24 17:51:20.878282 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:20.878239 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:51:20.915896 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:20.915863 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh5gj\" (UniqueName: \"kubernetes.io/projected/a7fb3593-eead-458d-a105-ad939a220cad-kube-api-access-sh5gj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:20.916058 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:20.915903 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7fb3593-eead-458d-a105-ad939a220cad-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:20.916058 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:20.915957 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7fb3593-eead-458d-a105-ad939a220cad-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:20.916058 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:20.916002 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7fb3593-eead-458d-a105-ad939a220cad-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:20.916058 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:20.916038 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7fb3593-eead-458d-a105-ad939a220cad-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:20.916214 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:20.916073 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7fb3593-eead-458d-a105-ad939a220cad-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:21.016699 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.016613 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7fb3593-eead-458d-a105-ad939a220cad-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:21.016699 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.016652 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7fb3593-eead-458d-a105-ad939a220cad-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:21.016877 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.016779 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7fb3593-eead-458d-a105-ad939a220cad-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:21.016877 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.016855 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7fb3593-eead-458d-a105-ad939a220cad-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:21.016984 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.016906 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh5gj\" (UniqueName: \"kubernetes.io/projected/a7fb3593-eead-458d-a105-ad939a220cad-kube-api-access-sh5gj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:21.016984 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.016965 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7fb3593-eead-458d-a105-ad939a220cad-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:21.017068 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.017006 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7fb3593-eead-458d-a105-ad939a220cad-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:21.017114 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.017092 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7fb3593-eead-458d-a105-ad939a220cad-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:21.017350 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.017331 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7fb3593-eead-458d-a105-ad939a220cad-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:21.018975 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.018932 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7fb3593-eead-458d-a105-ad939a220cad-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:21.019231 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.019214 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7fb3593-eead-458d-a105-ad939a220cad-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:21.026307 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.026288 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh5gj\" (UniqueName: \"kubernetes.io/projected/a7fb3593-eead-458d-a105-ad939a220cad-kube-api-access-sh5gj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8zszk\" (UID: \"a7fb3593-eead-458d-a105-ad939a220cad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:21.077789 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.077752 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:21.218169 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.218148 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk"] Jul 24 17:51:21.220046 ip-10-0-137-185 kubenswrapper[2561]: W0724 17:51:21.220017 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7fb3593_eead_458d_a105_ad939a220cad.slice/crio-3ecb8b7f04f987c965affdd39e93b4960f46b77da3b3526906c937792aef7596 WatchSource:0}: Error finding container 3ecb8b7f04f987c965affdd39e93b4960f46b77da3b3526906c937792aef7596: Status 404 returned error can't find the container with id 3ecb8b7f04f987c965affdd39e93b4960f46b77da3b3526906c937792aef7596 Jul 24 17:51:21.221194 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.221180 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 17:51:21.463335 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.463308 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" event={"ID":"a7fb3593-eead-458d-a105-ad939a220cad","Type":"ContainerStarted","Data":"8626ffedfee3d6b921e16e9a0f46128a81a7a9d05260be04b9cacac6caca8004"} Jul 24 17:51:21.463480 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.463358 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" event={"ID":"a7fb3593-eead-458d-a105-ad939a220cad","Type":"ContainerStarted","Data":"3ecb8b7f04f987c965affdd39e93b4960f46b77da3b3526906c937792aef7596"} Jul 24 17:51:21.510174 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:21.510120 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" podStartSLOduration=1.33326926 podStartE2EDuration="1.510104647s" totalImagesPullingTime="176.835387ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:51:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:51:21.508252491 +0000 UTC m=+677.121258360" watchObservedRunningTime="2026-07-24 17:51:21.510104647 +0000 UTC m=+677.123110513" Jul 24 17:51:31.077882 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:31.077843 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:31.078281 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:31.077901 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:31.090366 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:31.090341 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:31.511875 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:31.511849 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8zszk" Jul 24 17:51:46.440623 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:51:46.440576 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:52:30.000707 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:52:30.000626 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:52:30.062416 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:52:30.062377 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:52:42.569626 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:52:42.569591 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:52:42.623672 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:52:42.623622 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:52:43.770092 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:52:43.770048 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:52:43.822184 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:52:43.822146 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:53:33.572161 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:53:33.572078 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:53:33.631932 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:53:33.631893 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:53:41.740180 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:53:41.740144 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:53:41.828674 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:53:41.828634 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:53:43.006635 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:53:43.006594 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:53:43.138422 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:53:43.138385 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:55:33.058926 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:55:33.058893 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:55:51.265274 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:55:51.265241 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:55:52.765433 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:55:52.765397 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:56:27.650157 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:56:27.650084 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:56:27.781729 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:56:27.781696 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:56:27.883317 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:56:27.883281 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:56:33.646953 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:56:33.646895 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:56:33.747536 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:56:33.747510 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:56:33.870296 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:56:33.870263 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:56:33.976731 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:56:33.976701 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:56:34.104290 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:56:34.104259 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:56:39.859265 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:56:39.859229 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:56:39.912779 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:56:39.912751 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:57:59.583973 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:57:59.583919 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 17:59:32.869665 ip-10-0-137-185 kubenswrapper[2561]: I0724 17:59:32.869586 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:00:00.144651 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:00:00.144613 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748600-gms8p"] Jul 24 18:00:00.147710 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:00:00.147692 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-gms8p" Jul 24 18:00:00.150283 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:00:00.150266 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\"" Jul 24 18:00:00.156772 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:00:00.156754 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-gms8p" Jul 24 18:00:00.165234 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:00:00.163327 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748600-gms8p"] Jul 24 18:00:00.292242 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:00:00.292215 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748600-gms8p"] Jul 24 18:00:00.295160 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:00:00.295130 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod886ddf63_c505_41e1_989d_994c58a56bea.slice/crio-4b100e73dc57ff0a9df9781efd2b3da690a4f408993153d5cab9eaa8413b636a WatchSource:0}: Error finding container 4b100e73dc57ff0a9df9781efd2b3da690a4f408993153d5cab9eaa8413b636a: Status 404 returned error can't find the container with id 4b100e73dc57ff0a9df9781efd2b3da690a4f408993153d5cab9eaa8413b636a Jul 24 18:00:00.297276 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:00:00.297258 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 18:00:00.504607 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:00:00.504573 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-gms8p" event={"ID":"886ddf63-c505-41e1-989d-994c58a56bea","Type":"ContainerStarted","Data":"4b100e73dc57ff0a9df9781efd2b3da690a4f408993153d5cab9eaa8413b636a"} Jul 24 18:00:03.521341 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:00:03.521304 2561 generic.go:356] "Generic (PLEG): container finished" podID="886ddf63-c505-41e1-989d-994c58a56bea" containerID="8b990998dc8b84208348f9227ed10e4bcf8c520ff0c6871457528c400d4d46b8" exitCode=0 Jul 24 18:00:03.521711 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:00:03.521386 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-gms8p" event={"ID":"886ddf63-c505-41e1-989d-994c58a56bea","Type":"ContainerDied","Data":"8b990998dc8b84208348f9227ed10e4bcf8c520ff0c6871457528c400d4d46b8"} Jul 24 18:00:04.651690 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:00:04.651663 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-gms8p" Jul 24 18:00:05.530830 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:00:05.530804 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-gms8p" Jul 24 18:00:05.531048 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:00:05.530808 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-gms8p" event={"ID":"886ddf63-c505-41e1-989d-994c58a56bea","Type":"ContainerDied","Data":"4b100e73dc57ff0a9df9781efd2b3da690a4f408993153d5cab9eaa8413b636a"} Jul 24 18:00:05.531048 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:00:05.530916 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b100e73dc57ff0a9df9781efd2b3da690a4f408993153d5cab9eaa8413b636a" Jul 24 18:02:51.469231 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:02:51.469189 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:02:52.734107 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:02:52.734072 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:04:16.583886 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:04:16.583806 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:04:18.796803 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:04:18.796772 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:04:36.143874 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:04:36.143840 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:04:44.129340 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:04:44.129302 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:04:44.230196 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:04:44.230160 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:05:06.837877 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:05:06.837842 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:05:07.184839 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:05:07.184807 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:07:32.014616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:07:32.014540 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:15.464143 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:15.464112 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:39.305461 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:39.305425 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:42.720783 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:42.720748 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:42.821244 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:42.821211 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:43.225425 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:43.225395 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:49.032378 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.032337 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:49.110651 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.110620 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:49.302885 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.302810 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:49.597416 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.597339 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5"] Jul 24 18:08:49.597759 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.597746 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="886ddf63-c505-41e1-989d-994c58a56bea" containerName="cleanup" Jul 24 18:08:49.597800 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.597763 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="886ddf63-c505-41e1-989d-994c58a56bea" containerName="cleanup" Jul 24 18:08:49.597861 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.597851 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="886ddf63-c505-41e1-989d-994c58a56bea" containerName="cleanup" Jul 24 18:08:49.601108 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.601085 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.607705 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.607684 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3e8e6244-openshift-default-dockercfg-mv84g\"" Jul 24 18:08:49.640550 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.640512 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:49.654122 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.654101 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5"] Jul 24 18:08:49.677615 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.677582 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-data\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.677720 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.677628 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-credential-socket\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.677720 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.677663 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-envoy\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.677720 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.677699 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-token\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.677822 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.677719 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2bj4\" (UniqueName: \"kubernetes.io/projected/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-kube-api-access-c2bj4\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.677822 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.677759 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-workload-certs\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.677822 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.677786 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istiod-ca-cert\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.677822 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.677807 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-workload-socket\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.677978 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.677847 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-podinfo\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.778461 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.778432 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-token\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.778461 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.778464 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2bj4\" (UniqueName: \"kubernetes.io/projected/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-kube-api-access-c2bj4\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.778672 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.778485 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-workload-certs\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.778672 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.778525 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istiod-ca-cert\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.778672 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.778556 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-workload-socket\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.778672 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.778614 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-podinfo\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.778672 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.778648 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-data\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.779052 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.778684 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-credential-socket\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.779052 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.778723 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-envoy\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.779052 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.778960 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-workload-certs\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.779210 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.779069 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-workload-socket\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.779210 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.779081 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-data\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.779210 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.779117 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-credential-socket\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.779399 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.779377 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istiod-ca-cert\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.781022 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.781000 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-envoy\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.781122 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.781108 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-podinfo\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.788173 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.788150 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-token\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.788768 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.788745 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2bj4\" (UniqueName: \"kubernetes.io/projected/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-kube-api-access-c2bj4\") pod \"e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:49.808650 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.808630 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:49.910714 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:49.910645 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:50.040725 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:50.040702 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5"] Jul 24 18:08:50.042693 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:08:50.042663 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f85db9c_13b7_4292_8a9c_9a07d4d02a13.slice/crio-7c73f68b83327ab7836b9d6cc61dad5ac91852141101de4b6ddd6fe221a25cc3 WatchSource:0}: Error finding container 7c73f68b83327ab7836b9d6cc61dad5ac91852141101de4b6ddd6fe221a25cc3: Status 404 returned error can't find the container with id 7c73f68b83327ab7836b9d6cc61dad5ac91852141101de4b6ddd6fe221a25cc3 Jul 24 18:08:50.044707 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:50.044677 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:08:50.044758 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:50.044741 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:08:50.044794 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:50.044771 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:08:50.621989 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:50.621954 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" event={"ID":"0f85db9c-13b7-4292-8a9c-9a07d4d02a13","Type":"ContainerStarted","Data":"40cc8186bf0afa4721a0a0754626c07317a5168ffe7be4e04c2a1d6fbe32326c"} Jul 24 18:08:50.622155 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:50.622007 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" event={"ID":"0f85db9c-13b7-4292-8a9c-9a07d4d02a13","Type":"ContainerStarted","Data":"7c73f68b83327ab7836b9d6cc61dad5ac91852141101de4b6ddd6fe221a25cc3"} Jul 24 18:08:50.643067 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:50.643022 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" podStartSLOduration=1.6430082700000002 podStartE2EDuration="1.64300827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:08:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:08:50.641299021 +0000 UTC m=+1726.254304888" watchObservedRunningTime="2026-07-24 18:08:50.64300827 +0000 UTC m=+1726.256014188" Jul 24 18:08:50.911689 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:50.911605 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:51.916178 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:51.916144 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:52.077642 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:52.077611 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:52.176768 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:52.176696 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:52.264336 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:52.264304 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:52.335116 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:52.335082 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:52.629921 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:52.629893 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:52.630765 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:52.630750 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:08:52.904106 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:52.904018 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:08:52.988933 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:08:52.988899 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:07.631894 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:07.631864 2561 patch_prober.go:28] interesting pod/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Jul 24 18:09:07.632374 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:07.631953 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" podUID="0f85db9c-13b7-4292-8a9c-9a07d4d02a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Jul 24 18:09:07.691659 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:07.691625 2561 generic.go:356] "Generic (PLEG): container finished" podID="0f85db9c-13b7-4292-8a9c-9a07d4d02a13" containerID="40cc8186bf0afa4721a0a0754626c07317a5168ffe7be4e04c2a1d6fbe32326c" exitCode=0 Jul 24 18:09:07.691812 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:07.691695 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" event={"ID":"0f85db9c-13b7-4292-8a9c-9a07d4d02a13","Type":"ContainerDied","Data":"40cc8186bf0afa4721a0a0754626c07317a5168ffe7be4e04c2a1d6fbe32326c"} Jul 24 18:09:07.692161 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:07.692142 2561 scope.go:117] "RemoveContainer" containerID="40cc8186bf0afa4721a0a0754626c07317a5168ffe7be4e04c2a1d6fbe32326c" Jul 24 18:09:07.693174 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:07.693147 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:09:07.693244 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:07.693214 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:09:07.693285 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:07.693246 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:09:08.698820 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:08.698780 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" event={"ID":"0f85db9c-13b7-4292-8a9c-9a07d4d02a13","Type":"ContainerStarted","Data":"3b97c84890e7e6967fe01d5d44b25ef0948baddce61fe663b9c352a86a97a60a"} Jul 24 18:09:08.911582 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:08.911542 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:09:08.911798 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:08.911776 2561 patch_prober.go:28] interesting pod/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Jul 24 18:09:08.911855 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:08.911836 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" podUID="0f85db9c-13b7-4292-8a9c-9a07d4d02a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Jul 24 18:09:09.911686 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:09.911647 2561 patch_prober.go:28] interesting pod/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Jul 24 18:09:09.912123 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:09.911706 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" podUID="0f85db9c-13b7-4292-8a9c-9a07d4d02a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Jul 24 18:09:10.912080 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:10.912046 2561 patch_prober.go:28] interesting pod/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Jul 24 18:09:10.912475 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:10.912109 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" podUID="0f85db9c-13b7-4292-8a9c-9a07d4d02a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Jul 24 18:09:11.911758 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:11.911721 2561 patch_prober.go:28] interesting pod/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Jul 24 18:09:11.911908 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:11.911777 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" podUID="0f85db9c-13b7-4292-8a9c-9a07d4d02a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Jul 24 18:09:12.915336 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:12.915302 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:09:12.915763 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:12.915559 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:09:12.916139 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:12.916122 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:09:17.670112 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:17.670078 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr"] Jul 24 18:09:17.672188 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:17.672170 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr" Jul 24 18:09:17.674785 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:17.674749 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 18:09:17.675136 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:17.675115 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\"" Jul 24 18:09:17.684909 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:17.684885 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr"] Jul 24 18:09:17.707827 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:17.707799 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6505596-b7e6-4c4e-ac5b-13dfbac5ed33-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr\" (UID: \"a6505596-b7e6-4c4e-ac5b-13dfbac5ed33\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr" Jul 24 18:09:17.808336 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:17.808308 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6505596-b7e6-4c4e-ac5b-13dfbac5ed33-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr\" (UID: \"a6505596-b7e6-4c4e-ac5b-13dfbac5ed33\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr" Jul 24 18:09:17.808855 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:17.808838 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6505596-b7e6-4c4e-ac5b-13dfbac5ed33-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr\" (UID: \"a6505596-b7e6-4c4e-ac5b-13dfbac5ed33\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr" Jul 24 18:09:17.981899 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:17.981862 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr" Jul 24 18:09:18.105667 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:18.105628 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr"] Jul 24 18:09:18.107824 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:09:18.107795 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6505596_b7e6_4c4e_ac5b_13dfbac5ed33.slice/crio-8a40fd0a69b8dc12c4b566da82364809e5a38c63f09402292b8ad2dddd27a829 WatchSource:0}: Error finding container 8a40fd0a69b8dc12c4b566da82364809e5a38c63f09402292b8ad2dddd27a829: Status 404 returned error can't find the container with id 8a40fd0a69b8dc12c4b566da82364809e5a38c63f09402292b8ad2dddd27a829 Jul 24 18:09:18.739779 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:18.739746 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr" event={"ID":"a6505596-b7e6-4c4e-ac5b-13dfbac5ed33","Type":"ContainerStarted","Data":"911f5e5acc219db6fce170e14cd90a35fc63bd41e9f6495c008f03430d0ca84c"} Jul 24 18:09:18.740254 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:18.739806 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr" event={"ID":"a6505596-b7e6-4c4e-ac5b-13dfbac5ed33","Type":"ContainerStarted","Data":"8a40fd0a69b8dc12c4b566da82364809e5a38c63f09402292b8ad2dddd27a829"} Jul 24 18:09:18.756848 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:18.756801 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr" podStartSLOduration=1.756786134 podStartE2EDuration="1.756786134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:09:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:09:18.755358601 +0000 UTC m=+1754.368364466" watchObservedRunningTime="2026-07-24 18:09:18.756786134 +0000 UTC m=+1754.369792002" Jul 24 18:09:19.744896 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:19.744857 2561 generic.go:356] "Generic (PLEG): container finished" podID="a6505596-b7e6-4c4e-ac5b-13dfbac5ed33" containerID="911f5e5acc219db6fce170e14cd90a35fc63bd41e9f6495c008f03430d0ca84c" exitCode=0 Jul 24 18:09:19.745490 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:19.744948 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr" event={"ID":"a6505596-b7e6-4c4e-ac5b-13dfbac5ed33","Type":"ContainerDied","Data":"911f5e5acc219db6fce170e14cd90a35fc63bd41e9f6495c008f03430d0ca84c"} Jul 24 18:09:20.871327 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:20.871303 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr" Jul 24 18:09:20.935986 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:20.935931 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6505596-b7e6-4c4e-ac5b-13dfbac5ed33-maas-api-service-ca\") pod \"a6505596-b7e6-4c4e-ac5b-13dfbac5ed33\" (UID: \"a6505596-b7e6-4c4e-ac5b-13dfbac5ed33\") " Jul 24 18:09:20.936266 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:20.936243 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6505596-b7e6-4c4e-ac5b-13dfbac5ed33-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a6505596-b7e6-4c4e-ac5b-13dfbac5ed33" (UID: "a6505596-b7e6-4c4e-ac5b-13dfbac5ed33"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:09:21.037554 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:21.037478 2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6505596-b7e6-4c4e-ac5b-13dfbac5ed33-maas-api-service-ca\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:09:21.753954 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:21.753914 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr" event={"ID":"a6505596-b7e6-4c4e-ac5b-13dfbac5ed33","Type":"ContainerDied","Data":"8a40fd0a69b8dc12c4b566da82364809e5a38c63f09402292b8ad2dddd27a829"} Jul 24 18:09:21.754100 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:21.753968 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr" Jul 24 18:09:21.754100 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:21.753984 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a40fd0a69b8dc12c4b566da82364809e5a38c63f09402292b8ad2dddd27a829" Jul 24 18:09:28.216397 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:28.216360 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:28.287027 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:28.286994 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:28.350787 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:28.350756 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:32.749060 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:32.749024 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr"] Jul 24 18:09:32.751074 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:32.751051 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3e8e6244-hv7hr"] Jul 24 18:09:32.752915 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:32.752893 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:32.894151 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:32.894117 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5"] Jul 24 18:09:32.894408 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:32.894381 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" podUID="0f85db9c-13b7-4292-8a9c-9a07d4d02a13" containerName="istio-proxy" containerID="cri-o://3b97c84890e7e6967fe01d5d44b25ef0948baddce61fe663b9c352a86a97a60a" gracePeriod=30 Jul 24 18:09:32.911575 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:32.911543 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:32.939679 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:32.939643 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6505596-b7e6-4c4e-ac5b-13dfbac5ed33" path="/var/lib/kubelet/pods/a6505596-b7e6-4c4e-ac5b-13dfbac5ed33/volumes" Jul 24 18:09:33.164032 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:33.163933 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:33.398222 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:33.398192 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:33.489879 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:33.489847 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:33.609132 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:33.609097 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:33.907664 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:33.907575 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw"] Jul 24 18:09:33.908107 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:33.908038 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6505596-b7e6-4c4e-ac5b-13dfbac5ed33" containerName="revoke-keys" Jul 24 18:09:33.908107 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:33.908058 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6505596-b7e6-4c4e-ac5b-13dfbac5ed33" containerName="revoke-keys" Jul 24 18:09:33.908219 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:33.908150 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6505596-b7e6-4c4e-ac5b-13dfbac5ed33" containerName="revoke-keys" Jul 24 18:09:33.910051 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:33.910020 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:33.912283 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:33.912267 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-8be87499-gw-openshift-default-dockercfg-rfw4g\"" Jul 24 18:09:33.923322 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:33.923299 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw"] Jul 24 18:09:33.932085 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:33.932058 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:34.031609 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.031578 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:34.047452 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.047425 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20122220-13bf-4cec-a1fc-d90cd374462e-istio-token\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.047538 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.047464 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-workload-socket\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.047583 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.047551 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-credential-socket\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.047623 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.047600 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20122220-13bf-4cec-a1fc-d90cd374462e-istio-podinfo\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.047655 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.047623 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-istio-envoy\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.047655 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.047650 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-istio-data\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.047716 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.047682 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20122220-13bf-4cec-a1fc-d90cd374462e-istiod-ca-cert\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.047716 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.047709 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-workload-certs\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.047779 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.047734 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fqnj\" (UniqueName: \"kubernetes.io/projected/20122220-13bf-4cec-a1fc-d90cd374462e-kube-api-access-9fqnj\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.148476 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.148444 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-istio-envoy\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.148652 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.148487 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-istio-data\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.148652 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.148511 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20122220-13bf-4cec-a1fc-d90cd374462e-istiod-ca-cert\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.148652 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.148537 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-workload-certs\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.148652 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.148572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fqnj\" (UniqueName: \"kubernetes.io/projected/20122220-13bf-4cec-a1fc-d90cd374462e-kube-api-access-9fqnj\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.148652 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.148603 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20122220-13bf-4cec-a1fc-d90cd374462e-istio-token\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.148652 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.148644 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-workload-socket\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.149066 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.148717 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-credential-socket\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.149066 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.148770 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20122220-13bf-4cec-a1fc-d90cd374462e-istio-podinfo\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.149066 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.148993 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-istio-data\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.149066 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.149035 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-workload-certs\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.149256 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.149233 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-credential-socket\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.149321 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.149267 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-workload-socket\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.149371 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.149335 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20122220-13bf-4cec-a1fc-d90cd374462e-istiod-ca-cert\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.150849 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.150822 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-istio-envoy\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.151156 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.151140 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20122220-13bf-4cec-a1fc-d90cd374462e-istio-podinfo\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.156187 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.156165 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fqnj\" (UniqueName: \"kubernetes.io/projected/20122220-13bf-4cec-a1fc-d90cd374462e-kube-api-access-9fqnj\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.156759 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.156738 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20122220-13bf-4cec-a1fc-d90cd374462e-istio-token\") pod \"e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.220986 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.220968 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:34.349778 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.349665 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw"] Jul 24 18:09:34.351843 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:09:34.351816 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20122220_13bf_4cec_a1fc_d90cd374462e.slice/crio-f303363fc3a36ef0afaef3619b3e446306996108528169e9f35964ab6fe667ff WatchSource:0}: Error finding container f303363fc3a36ef0afaef3619b3e446306996108528169e9f35964ab6fe667ff: Status 404 returned error can't find the container with id f303363fc3a36ef0afaef3619b3e446306996108528169e9f35964ab6fe667ff Jul 24 18:09:34.353982 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.353952 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:09:34.354052 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.354014 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:09:34.354052 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.354046 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:09:34.809292 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.809250 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" event={"ID":"20122220-13bf-4cec-a1fc-d90cd374462e","Type":"ContainerStarted","Data":"ec1da62c7fbc835af03ea47f336591c0c8cb5d5128d1e2c5671744db6867278f"} Jul 24 18:09:34.809464 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.809308 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" event={"ID":"20122220-13bf-4cec-a1fc-d90cd374462e","Type":"ContainerStarted","Data":"f303363fc3a36ef0afaef3619b3e446306996108528169e9f35964ab6fe667ff"} Jul 24 18:09:34.829879 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:34.829827 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" podStartSLOduration=1.829811233 podStartE2EDuration="1.829811233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:09:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:09:34.827911119 +0000 UTC m=+1770.440916987" watchObservedRunningTime="2026-07-24 18:09:34.829811233 +0000 UTC m=+1770.442817154" Jul 24 18:09:35.221681 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:35.221650 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:35.227100 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:35.227078 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:35.814311 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:35.814284 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:35.815383 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:35.815362 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:36.311803 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:36.311771 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:36.421994 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:36.421957 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:36.827265 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:36.827232 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:36.981313 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:36.981280 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:37.081008 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:37.080908 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:09:38.140870 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.140842 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:09:38.291559 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.291530 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-podinfo\") pod \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " Jul 24 18:09:38.291704 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.291572 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-workload-certs\") pod \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " Jul 24 18:09:38.291704 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.291600 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-token\") pod \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " Jul 24 18:09:38.291704 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.291649 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-workload-socket\") pod \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " Jul 24 18:09:38.291704 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.291680 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-credential-socket\") pod \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " Jul 24 18:09:38.291907 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.291741 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-envoy\") pod \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " Jul 24 18:09:38.291907 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.291790 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-data\") pod \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " Jul 24 18:09:38.291907 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.291823 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0f85db9c-13b7-4292-8a9c-9a07d4d02a13" (UID: "0f85db9c-13b7-4292-8a9c-9a07d4d02a13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:09:38.291907 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.291840 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2bj4\" (UniqueName: \"kubernetes.io/projected/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-kube-api-access-c2bj4\") pod \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " Jul 24 18:09:38.291907 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.291867 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istiod-ca-cert\") pod \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\" (UID: \"0f85db9c-13b7-4292-8a9c-9a07d4d02a13\") " Jul 24 18:09:38.292249 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.292159 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0f85db9c-13b7-4292-8a9c-9a07d4d02a13" (UID: "0f85db9c-13b7-4292-8a9c-9a07d4d02a13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:09:38.292249 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.292186 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:09:38.292354 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.292307 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0f85db9c-13b7-4292-8a9c-9a07d4d02a13" (UID: "0f85db9c-13b7-4292-8a9c-9a07d4d02a13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:09:38.292550 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.292530 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0f85db9c-13b7-4292-8a9c-9a07d4d02a13" (UID: "0f85db9c-13b7-4292-8a9c-9a07d4d02a13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:09:38.292851 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.292830 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-data" (OuterVolumeSpecName: "istio-data") pod "0f85db9c-13b7-4292-8a9c-9a07d4d02a13" (UID: "0f85db9c-13b7-4292-8a9c-9a07d4d02a13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:09:38.294034 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.294007 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0f85db9c-13b7-4292-8a9c-9a07d4d02a13" (UID: "0f85db9c-13b7-4292-8a9c-9a07d4d02a13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:09:38.294034 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.294021 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0f85db9c-13b7-4292-8a9c-9a07d4d02a13" (UID: "0f85db9c-13b7-4292-8a9c-9a07d4d02a13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:09:38.294235 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.294219 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-kube-api-access-c2bj4" (OuterVolumeSpecName: "kube-api-access-c2bj4") pod "0f85db9c-13b7-4292-8a9c-9a07d4d02a13" (UID: "0f85db9c-13b7-4292-8a9c-9a07d4d02a13"). InnerVolumeSpecName "kube-api-access-c2bj4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:09:38.294295 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.294235 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-token" (OuterVolumeSpecName: "istio-token") pod "0f85db9c-13b7-4292-8a9c-9a07d4d02a13" (UID: "0f85db9c-13b7-4292-8a9c-9a07d4d02a13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:09:38.393338 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.393308 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:09:38.393338 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.393330 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:09:38.393504 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.393343 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2bj4\" (UniqueName: \"kubernetes.io/projected/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-kube-api-access-c2bj4\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:09:38.393504 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.393357 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:09:38.393504 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.393368 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:09:38.393504 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.393378 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:09:38.393504 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.393389 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:09:38.393504 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.393401 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f85db9c-13b7-4292-8a9c-9a07d4d02a13-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:09:38.828953 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.828898 2561 generic.go:356] "Generic (PLEG): container finished" podID="0f85db9c-13b7-4292-8a9c-9a07d4d02a13" containerID="3b97c84890e7e6967fe01d5d44b25ef0948baddce61fe663b9c352a86a97a60a" exitCode=0 Jul 24 18:09:38.829117 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.828958 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" Jul 24 18:09:38.829117 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.828984 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" event={"ID":"0f85db9c-13b7-4292-8a9c-9a07d4d02a13","Type":"ContainerDied","Data":"3b97c84890e7e6967fe01d5d44b25ef0948baddce61fe663b9c352a86a97a60a"} Jul 24 18:09:38.829117 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.829044 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5" event={"ID":"0f85db9c-13b7-4292-8a9c-9a07d4d02a13","Type":"ContainerDied","Data":"7c73f68b83327ab7836b9d6cc61dad5ac91852141101de4b6ddd6fe221a25cc3"} Jul 24 18:09:38.829117 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.829068 2561 scope.go:117] "RemoveContainer" containerID="3b97c84890e7e6967fe01d5d44b25ef0948baddce61fe663b9c352a86a97a60a" Jul 24 18:09:38.837877 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.837861 2561 scope.go:117] "RemoveContainer" containerID="40cc8186bf0afa4721a0a0754626c07317a5168ffe7be4e04c2a1d6fbe32326c" Jul 24 18:09:38.845203 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.845184 2561 scope.go:117] "RemoveContainer" containerID="3b97c84890e7e6967fe01d5d44b25ef0948baddce61fe663b9c352a86a97a60a" Jul 24 18:09:38.845448 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:09:38.845426 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b97c84890e7e6967fe01d5d44b25ef0948baddce61fe663b9c352a86a97a60a\": container with ID starting with 3b97c84890e7e6967fe01d5d44b25ef0948baddce61fe663b9c352a86a97a60a not found: ID does not exist" containerID="3b97c84890e7e6967fe01d5d44b25ef0948baddce61fe663b9c352a86a97a60a" Jul 24 18:09:38.845497 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.845456 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b97c84890e7e6967fe01d5d44b25ef0948baddce61fe663b9c352a86a97a60a"} err="failed to get container status \"3b97c84890e7e6967fe01d5d44b25ef0948baddce61fe663b9c352a86a97a60a\": rpc error: code = NotFound desc = could not find container \"3b97c84890e7e6967fe01d5d44b25ef0948baddce61fe663b9c352a86a97a60a\": container with ID starting with 3b97c84890e7e6967fe01d5d44b25ef0948baddce61fe663b9c352a86a97a60a not found: ID does not exist" Jul 24 18:09:38.845497 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.845473 2561 scope.go:117] "RemoveContainer" containerID="40cc8186bf0afa4721a0a0754626c07317a5168ffe7be4e04c2a1d6fbe32326c" Jul 24 18:09:38.845668 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:09:38.845652 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40cc8186bf0afa4721a0a0754626c07317a5168ffe7be4e04c2a1d6fbe32326c\": container with ID starting with 40cc8186bf0afa4721a0a0754626c07317a5168ffe7be4e04c2a1d6fbe32326c not found: ID does not exist" containerID="40cc8186bf0afa4721a0a0754626c07317a5168ffe7be4e04c2a1d6fbe32326c" Jul 24 18:09:38.845707 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.845674 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40cc8186bf0afa4721a0a0754626c07317a5168ffe7be4e04c2a1d6fbe32326c"} err="failed to get container status \"40cc8186bf0afa4721a0a0754626c07317a5168ffe7be4e04c2a1d6fbe32326c\": rpc error: code = NotFound desc = could not find container \"40cc8186bf0afa4721a0a0754626c07317a5168ffe7be4e04c2a1d6fbe32326c\": container with ID starting with 40cc8186bf0afa4721a0a0754626c07317a5168ffe7be4e04c2a1d6fbe32326c not found: ID does not exist" Jul 24 18:09:38.849948 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.849915 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5"] Jul 24 18:09:38.854048 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.854030 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3e8e6244-openshift-default-756c44b998-rv2l5"] Jul 24 18:09:38.946366 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:38.946322 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f85db9c-13b7-4292-8a9c-9a07d4d02a13" path="/var/lib/kubelet/pods/0f85db9c-13b7-4292-8a9c-9a07d4d02a13/volumes" Jul 24 18:09:50.816040 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:50.816004 2561 patch_prober.go:28] interesting pod/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 24 18:09:50.816434 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:50.816060 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" podUID="20122220-13bf-4cec-a1fc-d90cd374462e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 24 18:09:50.879165 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:50.879135 2561 generic.go:356] "Generic (PLEG): container finished" podID="20122220-13bf-4cec-a1fc-d90cd374462e" containerID="ec1da62c7fbc835af03ea47f336591c0c8cb5d5128d1e2c5671744db6867278f" exitCode=0 Jul 24 18:09:50.879313 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:50.879209 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" event={"ID":"20122220-13bf-4cec-a1fc-d90cd374462e","Type":"ContainerDied","Data":"ec1da62c7fbc835af03ea47f336591c0c8cb5d5128d1e2c5671744db6867278f"} Jul 24 18:09:50.879613 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:50.879600 2561 scope.go:117] "RemoveContainer" containerID="ec1da62c7fbc835af03ea47f336591c0c8cb5d5128d1e2c5671744db6867278f" Jul 24 18:09:50.880672 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:50.880648 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:09:50.880731 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:50.880711 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:09:50.880766 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:50.880742 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:09:51.887916 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:51.887876 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" event={"ID":"20122220-13bf-4cec-a1fc-d90cd374462e","Type":"ContainerStarted","Data":"b9a3940a442ebace9e9f211719715f85b0472969bd1d6af2ee98c2f3dc2263ca"} Jul 24 18:09:52.221536 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:52.221497 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:52.221760 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:52.221737 2561 patch_prober.go:28] interesting pod/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 24 18:09:52.221810 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:52.221786 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" podUID="20122220-13bf-4cec-a1fc-d90cd374462e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 24 18:09:53.221554 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:53.221516 2561 patch_prober.go:28] interesting pod/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 24 18:09:53.221979 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:53.221585 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" podUID="20122220-13bf-4cec-a1fc-d90cd374462e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 24 18:09:54.222208 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:54.222170 2561 patch_prober.go:28] interesting pod/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 24 18:09:54.222582 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:54.222227 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" podUID="20122220-13bf-4cec-a1fc-d90cd374462e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 24 18:09:55.222153 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:55.222112 2561 patch_prober.go:28] interesting pod/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 24 18:09:55.222323 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:55.222182 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" podUID="20122220-13bf-4cec-a1fc-d90cd374462e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 24 18:09:56.225650 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:56.225621 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:56.226041 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:56.225860 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:09:56.226447 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:09:56.226428 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:10:01.397179 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.397101 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8be87499-q677w"] Jul 24 18:10:01.397608 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.397393 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f85db9c-13b7-4292-8a9c-9a07d4d02a13" containerName="istio-proxy" Jul 24 18:10:01.397608 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.397404 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f85db9c-13b7-4292-8a9c-9a07d4d02a13" containerName="istio-proxy" Jul 24 18:10:01.397608 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.397419 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f85db9c-13b7-4292-8a9c-9a07d4d02a13" containerName="istio-proxy" Jul 24 18:10:01.397608 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.397425 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f85db9c-13b7-4292-8a9c-9a07d4d02a13" containerName="istio-proxy" Jul 24 18:10:01.397608 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.397471 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f85db9c-13b7-4292-8a9c-9a07d4d02a13" containerName="istio-proxy" Jul 24 18:10:01.397608 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.397477 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f85db9c-13b7-4292-8a9c-9a07d4d02a13" containerName="istio-proxy" Jul 24 18:10:01.400308 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.400280 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8be87499-q677w" Jul 24 18:10:01.403236 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.403209 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8be87499-q677w"] Jul 24 18:10:01.404075 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.404051 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\"" Jul 24 18:10:01.404190 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.404051 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 18:10:01.488885 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.488855 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d656f98-101b-4847-a2af-494ea41f9cea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8be87499-q677w\" (UID: \"4d656f98-101b-4847-a2af-494ea41f9cea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8be87499-q677w" Jul 24 18:10:01.589837 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.589808 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d656f98-101b-4847-a2af-494ea41f9cea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8be87499-q677w\" (UID: \"4d656f98-101b-4847-a2af-494ea41f9cea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8be87499-q677w" Jul 24 18:10:01.590419 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.590398 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d656f98-101b-4847-a2af-494ea41f9cea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8be87499-q677w\" (UID: \"4d656f98-101b-4847-a2af-494ea41f9cea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8be87499-q677w" Jul 24 18:10:01.712280 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.712252 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8be87499-q677w" Jul 24 18:10:01.842023 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.842000 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8be87499-q677w"] Jul 24 18:10:01.844419 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:10:01.844388 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d656f98_101b_4847_a2af_494ea41f9cea.slice/crio-ef1c0c8c74b3fd15c3383547e4ed19d57700553a48e1299ee3288c41c56b98f7 WatchSource:0}: Error finding container ef1c0c8c74b3fd15c3383547e4ed19d57700553a48e1299ee3288c41c56b98f7: Status 404 returned error can't find the container with id ef1c0c8c74b3fd15c3383547e4ed19d57700553a48e1299ee3288c41c56b98f7 Jul 24 18:10:01.933516 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:01.933484 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8be87499-q677w" event={"ID":"4d656f98-101b-4847-a2af-494ea41f9cea","Type":"ContainerStarted","Data":"ef1c0c8c74b3fd15c3383547e4ed19d57700553a48e1299ee3288c41c56b98f7"} Jul 24 18:10:02.939105 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:02.939075 2561 generic.go:356] "Generic (PLEG): container finished" podID="4d656f98-101b-4847-a2af-494ea41f9cea" containerID="ea209a1679664cb9f56060a874200008d9f07ec6a81448649cc5e715a055da15" exitCode=0 Jul 24 18:10:02.940453 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:02.940429 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8be87499-q677w" event={"ID":"4d656f98-101b-4847-a2af-494ea41f9cea","Type":"ContainerDied","Data":"ea209a1679664cb9f56060a874200008d9f07ec6a81448649cc5e715a055da15"} Jul 24 18:10:04.072209 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:04.072185 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8be87499-q677w" Jul 24 18:10:04.107003 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:04.106978 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d656f98-101b-4847-a2af-494ea41f9cea-maas-api-service-ca\") pod \"4d656f98-101b-4847-a2af-494ea41f9cea\" (UID: \"4d656f98-101b-4847-a2af-494ea41f9cea\") " Jul 24 18:10:04.107285 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:04.107266 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d656f98-101b-4847-a2af-494ea41f9cea-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d656f98-101b-4847-a2af-494ea41f9cea" (UID: "4d656f98-101b-4847-a2af-494ea41f9cea"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:10:04.207750 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:04.207690 2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d656f98-101b-4847-a2af-494ea41f9cea-maas-api-service-ca\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:10:04.948522 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:04.948498 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8be87499-q677w" Jul 24 18:10:04.948635 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:04.948522 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8be87499-q677w" event={"ID":"4d656f98-101b-4847-a2af-494ea41f9cea","Type":"ContainerDied","Data":"ef1c0c8c74b3fd15c3383547e4ed19d57700553a48e1299ee3288c41c56b98f7"} Jul 24 18:10:04.948635 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:04.948565 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef1c0c8c74b3fd15c3383547e4ed19d57700553a48e1299ee3288c41c56b98f7" Jul 24 18:10:11.848337 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:11.848303 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:11.929732 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:11.929696 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:12.027067 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:12.027037 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:16.464611 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:16.464573 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:16.471832 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:16.471804 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8be87499-q677w"] Jul 24 18:10:16.477075 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:16.477051 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8be87499-q677w"] Jul 24 18:10:16.609792 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:16.609760 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw"] Jul 24 18:10:16.610096 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:16.610055 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" podUID="20122220-13bf-4cec-a1fc-d90cd374462e" containerName="istio-proxy" containerID="cri-o://b9a3940a442ebace9e9f211719715f85b0472969bd1d6af2ee98c2f3dc2263ca" gracePeriod=30 Jul 24 18:10:16.635834 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:16.635805 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:16.869447 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:16.869364 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:16.939813 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:16.939783 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d656f98-101b-4847-a2af-494ea41f9cea" path="/var/lib/kubelet/pods/4d656f98-101b-4847-a2af-494ea41f9cea/volumes" Jul 24 18:10:17.161701 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.161620 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:17.427634 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.427552 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl"] Jul 24 18:10:17.427865 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.427853 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d656f98-101b-4847-a2af-494ea41f9cea" containerName="revoke-keys" Jul 24 18:10:17.427923 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.427866 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d656f98-101b-4847-a2af-494ea41f9cea" containerName="revoke-keys" Jul 24 18:10:17.427923 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.427920 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d656f98-101b-4847-a2af-494ea41f9cea" containerName="revoke-keys" Jul 24 18:10:17.430991 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.430969 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.433368 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.433348 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0858c257-openshift-default-dockercfg-gznb6\"" Jul 24 18:10:17.444582 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.444558 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl"] Jul 24 18:10:17.457587 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.457566 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:17.512525 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.512495 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnnw2\" (UniqueName: \"kubernetes.io/projected/fe54c62e-8695-4e50-8373-a4bcc09f1d81-kube-api-access-mnnw2\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.512849 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.512589 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-workload-socket\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.512849 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.512628 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-workload-certs\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.512849 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.512651 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-podinfo\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.512849 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.512680 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istiod-ca-cert\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.512849 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.512741 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-credential-socket\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.512849 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.512768 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-envoy\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.512849 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.512785 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-token\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.512849 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.512818 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-data\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.613888 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.613847 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-data\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.614090 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.613895 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnnw2\" (UniqueName: \"kubernetes.io/projected/fe54c62e-8695-4e50-8373-a4bcc09f1d81-kube-api-access-mnnw2\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.614090 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.613979 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-workload-socket\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.614090 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.614010 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-workload-certs\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.614257 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.614172 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-podinfo\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.614257 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.614250 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istiod-ca-cert\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.614362 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.614296 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-data\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.614362 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.614329 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-credential-socket\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.614362 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.614342 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-workload-certs\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.614508 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.614392 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-envoy\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.614508 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.614420 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-token\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.614597 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.614525 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-workload-socket\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.614646 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.614626 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-credential-socket\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.614953 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.614914 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istiod-ca-cert\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.616532 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.616511 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-envoy\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.616785 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.616764 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-podinfo\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.623087 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.623067 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-token\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.623521 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.623506 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnnw2\" (UniqueName: \"kubernetes.io/projected/fe54c62e-8695-4e50-8373-a4bcc09f1d81-kube-api-access-mnnw2\") pod \"e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.744240 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.744205 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:17.765156 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.765134 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:17.882496 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.882471 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl"] Jul 24 18:10:17.884856 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:10:17.884830 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe54c62e_8695_4e50_8373_a4bcc09f1d81.slice/crio-62419094a28bfc8939f95719fb2b284006e6fd557f63bb1f1194e8347c46f727 WatchSource:0}: Error finding container 62419094a28bfc8939f95719fb2b284006e6fd557f63bb1f1194e8347c46f727: Status 404 returned error can't find the container with id 62419094a28bfc8939f95719fb2b284006e6fd557f63bb1f1194e8347c46f727 Jul 24 18:10:17.886957 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.886899 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:10:17.887021 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.886994 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:10:17.887064 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:17.887025 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:10:18.005708 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:18.005625 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" event={"ID":"fe54c62e-8695-4e50-8373-a4bcc09f1d81","Type":"ContainerStarted","Data":"44414eabbae3f6335322ad338fb26a15d869abf940eebebcff2b7ee0cbe9c588"} Jul 24 18:10:18.005708 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:18.005682 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" event={"ID":"fe54c62e-8695-4e50-8373-a4bcc09f1d81","Type":"ContainerStarted","Data":"62419094a28bfc8939f95719fb2b284006e6fd557f63bb1f1194e8347c46f727"} Jul 24 18:10:18.025176 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:18.025123 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" podStartSLOduration=1.025103702 podStartE2EDuration="1.025103702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:10:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:10:18.023998616 +0000 UTC m=+1813.637004482" watchObservedRunningTime="2026-07-24 18:10:18.025103702 +0000 UTC m=+1813.638109570" Jul 24 18:10:18.744748 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:18.744714 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:18.749561 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:18.749540 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:19.010789 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:19.010706 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:19.011666 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:19.011649 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:19.383009 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:19.382906 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:19.484127 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:19.484102 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:19.583282 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:19.583239 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:19.779066 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:19.779033 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:20.173154 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:20.173068 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:20.262582 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:20.262552 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:21.855987 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.855964 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:10:21.953965 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.953920 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20122220-13bf-4cec-a1fc-d90cd374462e-istiod-ca-cert\") pod \"20122220-13bf-4cec-a1fc-d90cd374462e\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " Jul 24 18:10:21.954100 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.954000 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-istio-envoy\") pod \"20122220-13bf-4cec-a1fc-d90cd374462e\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " Jul 24 18:10:21.954100 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.954019 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-istio-data\") pod \"20122220-13bf-4cec-a1fc-d90cd374462e\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " Jul 24 18:10:21.954100 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.954044 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20122220-13bf-4cec-a1fc-d90cd374462e-istio-token\") pod \"20122220-13bf-4cec-a1fc-d90cd374462e\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " Jul 24 18:10:21.954100 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.954073 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20122220-13bf-4cec-a1fc-d90cd374462e-istio-podinfo\") pod \"20122220-13bf-4cec-a1fc-d90cd374462e\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " Jul 24 18:10:21.954293 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.954122 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-workload-socket\") pod \"20122220-13bf-4cec-a1fc-d90cd374462e\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " Jul 24 18:10:21.954293 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.954147 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-credential-socket\") pod \"20122220-13bf-4cec-a1fc-d90cd374462e\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " Jul 24 18:10:21.954293 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.954167 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-workload-certs\") pod \"20122220-13bf-4cec-a1fc-d90cd374462e\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " Jul 24 18:10:21.954293 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.954181 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fqnj\" (UniqueName: \"kubernetes.io/projected/20122220-13bf-4cec-a1fc-d90cd374462e-kube-api-access-9fqnj\") pod \"20122220-13bf-4cec-a1fc-d90cd374462e\" (UID: \"20122220-13bf-4cec-a1fc-d90cd374462e\") " Jul 24 18:10:21.954482 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.954382 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20122220-13bf-4cec-a1fc-d90cd374462e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "20122220-13bf-4cec-a1fc-d90cd374462e" (UID: "20122220-13bf-4cec-a1fc-d90cd374462e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:10:21.954482 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.954441 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "20122220-13bf-4cec-a1fc-d90cd374462e" (UID: "20122220-13bf-4cec-a1fc-d90cd374462e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:10:21.954657 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.954633 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "20122220-13bf-4cec-a1fc-d90cd374462e" (UID: "20122220-13bf-4cec-a1fc-d90cd374462e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:10:21.954841 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.954777 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "20122220-13bf-4cec-a1fc-d90cd374462e" (UID: "20122220-13bf-4cec-a1fc-d90cd374462e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:10:21.954975 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.954925 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-istio-data" (OuterVolumeSpecName: "istio-data") pod "20122220-13bf-4cec-a1fc-d90cd374462e" (UID: "20122220-13bf-4cec-a1fc-d90cd374462e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:10:21.956326 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.956297 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20122220-13bf-4cec-a1fc-d90cd374462e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "20122220-13bf-4cec-a1fc-d90cd374462e" (UID: "20122220-13bf-4cec-a1fc-d90cd374462e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:10:21.956419 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.956364 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "20122220-13bf-4cec-a1fc-d90cd374462e" (UID: "20122220-13bf-4cec-a1fc-d90cd374462e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:10:21.956419 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.956382 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20122220-13bf-4cec-a1fc-d90cd374462e-kube-api-access-9fqnj" (OuterVolumeSpecName: "kube-api-access-9fqnj") pod "20122220-13bf-4cec-a1fc-d90cd374462e" (UID: "20122220-13bf-4cec-a1fc-d90cd374462e"). InnerVolumeSpecName "kube-api-access-9fqnj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:10:21.956419 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:21.956403 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20122220-13bf-4cec-a1fc-d90cd374462e-istio-token" (OuterVolumeSpecName: "istio-token") pod "20122220-13bf-4cec-a1fc-d90cd374462e" (UID: "20122220-13bf-4cec-a1fc-d90cd374462e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:10:22.023552 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.023517 2561 generic.go:356] "Generic (PLEG): container finished" podID="20122220-13bf-4cec-a1fc-d90cd374462e" containerID="b9a3940a442ebace9e9f211719715f85b0472969bd1d6af2ee98c2f3dc2263ca" exitCode=0 Jul 24 18:10:22.023702 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.023573 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" Jul 24 18:10:22.023702 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.023608 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" event={"ID":"20122220-13bf-4cec-a1fc-d90cd374462e","Type":"ContainerDied","Data":"b9a3940a442ebace9e9f211719715f85b0472969bd1d6af2ee98c2f3dc2263ca"} Jul 24 18:10:22.023702 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.023664 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw" event={"ID":"20122220-13bf-4cec-a1fc-d90cd374462e","Type":"ContainerDied","Data":"f303363fc3a36ef0afaef3619b3e446306996108528169e9f35964ab6fe667ff"} Jul 24 18:10:22.023702 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.023686 2561 scope.go:117] "RemoveContainer" containerID="b9a3940a442ebace9e9f211719715f85b0472969bd1d6af2ee98c2f3dc2263ca" Jul 24 18:10:22.034409 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.034019 2561 scope.go:117] "RemoveContainer" containerID="ec1da62c7fbc835af03ea47f336591c0c8cb5d5128d1e2c5671744db6867278f" Jul 24 18:10:22.041500 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.041486 2561 scope.go:117] "RemoveContainer" containerID="b9a3940a442ebace9e9f211719715f85b0472969bd1d6af2ee98c2f3dc2263ca" Jul 24 18:10:22.041718 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:10:22.041701 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9a3940a442ebace9e9f211719715f85b0472969bd1d6af2ee98c2f3dc2263ca\": container with ID starting with b9a3940a442ebace9e9f211719715f85b0472969bd1d6af2ee98c2f3dc2263ca not found: ID does not exist" containerID="b9a3940a442ebace9e9f211719715f85b0472969bd1d6af2ee98c2f3dc2263ca" Jul 24 18:10:22.041760 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.041727 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9a3940a442ebace9e9f211719715f85b0472969bd1d6af2ee98c2f3dc2263ca"} err="failed to get container status \"b9a3940a442ebace9e9f211719715f85b0472969bd1d6af2ee98c2f3dc2263ca\": rpc error: code = NotFound desc = could not find container \"b9a3940a442ebace9e9f211719715f85b0472969bd1d6af2ee98c2f3dc2263ca\": container with ID starting with b9a3940a442ebace9e9f211719715f85b0472969bd1d6af2ee98c2f3dc2263ca not found: ID does not exist" Jul 24 18:10:22.041760 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.041743 2561 scope.go:117] "RemoveContainer" containerID="ec1da62c7fbc835af03ea47f336591c0c8cb5d5128d1e2c5671744db6867278f" Jul 24 18:10:22.041986 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:10:22.041964 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec1da62c7fbc835af03ea47f336591c0c8cb5d5128d1e2c5671744db6867278f\": container with ID starting with ec1da62c7fbc835af03ea47f336591c0c8cb5d5128d1e2c5671744db6867278f not found: ID does not exist" containerID="ec1da62c7fbc835af03ea47f336591c0c8cb5d5128d1e2c5671744db6867278f" Jul 24 18:10:22.042080 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.041988 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec1da62c7fbc835af03ea47f336591c0c8cb5d5128d1e2c5671744db6867278f"} err="failed to get container status \"ec1da62c7fbc835af03ea47f336591c0c8cb5d5128d1e2c5671744db6867278f\": rpc error: code = NotFound desc = could not find container \"ec1da62c7fbc835af03ea47f336591c0c8cb5d5128d1e2c5671744db6867278f\": container with ID starting with ec1da62c7fbc835af03ea47f336591c0c8cb5d5128d1e2c5671744db6867278f not found: ID does not exist" Jul 24 18:10:22.049335 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.049302 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw"] Jul 24 18:10:22.050747 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.050729 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-8be87499-gw-openshift-default-5f99dd5d56-rj4qw"] Jul 24 18:10:22.055165 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.055148 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20122220-13bf-4cec-a1fc-d90cd374462e-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:10:22.055210 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.055169 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:10:22.055210 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.055179 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:10:22.055210 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.055187 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:10:22.055210 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.055196 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fqnj\" (UniqueName: \"kubernetes.io/projected/20122220-13bf-4cec-a1fc-d90cd374462e-kube-api-access-9fqnj\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:10:22.055210 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.055203 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20122220-13bf-4cec-a1fc-d90cd374462e-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:10:22.055365 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.055212 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:10:22.055365 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.055220 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20122220-13bf-4cec-a1fc-d90cd374462e-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:10:22.055365 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.055228 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20122220-13bf-4cec-a1fc-d90cd374462e-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:10:22.940515 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:22.940482 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20122220-13bf-4cec-a1fc-d90cd374462e" path="/var/lib/kubelet/pods/20122220-13bf-4cec-a1fc-d90cd374462e/volumes" Jul 24 18:10:34.378159 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:34.377071 2561 patch_prober.go:28] interesting pod/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": read tcp 10.133.0.2:43438->10.133.0.28:15021: read: connection reset by peer" start-of-body= Jul 24 18:10:34.378159 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:34.377151 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" podUID="fe54c62e-8695-4e50-8373-a4bcc09f1d81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": read tcp 10.133.0.2:43438->10.133.0.28:15021: read: connection reset by peer" Jul 24 18:10:35.078208 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:35.078173 2561 generic.go:356] "Generic (PLEG): container finished" podID="fe54c62e-8695-4e50-8373-a4bcc09f1d81" containerID="44414eabbae3f6335322ad338fb26a15d869abf940eebebcff2b7ee0cbe9c588" exitCode=0 Jul 24 18:10:35.078442 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:35.078215 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" event={"ID":"fe54c62e-8695-4e50-8373-a4bcc09f1d81","Type":"ContainerDied","Data":"44414eabbae3f6335322ad338fb26a15d869abf940eebebcff2b7ee0cbe9c588"} Jul 24 18:10:35.078605 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:35.078590 2561 scope.go:117] "RemoveContainer" containerID="44414eabbae3f6335322ad338fb26a15d869abf940eebebcff2b7ee0cbe9c588" Jul 24 18:10:35.079576 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:35.079545 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:10:35.079681 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:35.079613 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:10:35.079681 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:35.079643 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:10:36.087306 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:36.087257 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" event={"ID":"fe54c62e-8695-4e50-8373-a4bcc09f1d81","Type":"ContainerStarted","Data":"2e994e52612f5088a1f63d7d00342867f4ec5c39bbdebd22ba0a1b390ae1293a"} Jul 24 18:10:36.744833 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:36.744794 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:36.745059 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:36.744960 2561 patch_prober.go:28] interesting pod/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 24 18:10:36.745137 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:36.745048 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" podUID="fe54c62e-8695-4e50-8373-a4bcc09f1d81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 24 18:10:37.744803 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:37.744770 2561 patch_prober.go:28] interesting pod/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 24 18:10:37.745202 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:37.744823 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" podUID="fe54c62e-8695-4e50-8373-a4bcc09f1d81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 24 18:10:38.745228 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:38.745188 2561 patch_prober.go:28] interesting pod/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 24 18:10:38.745600 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:38.745247 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" podUID="fe54c62e-8695-4e50-8373-a4bcc09f1d81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 24 18:10:39.745179 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:39.745141 2561 patch_prober.go:28] interesting pod/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 24 18:10:39.745361 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:39.745195 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" podUID="fe54c62e-8695-4e50-8373-a4bcc09f1d81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 24 18:10:40.749192 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:40.749154 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:40.749607 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:40.749509 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:40.750204 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:40.750188 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:10:45.272690 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.272656 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8"] Jul 24 18:10:45.273101 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.273030 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20122220-13bf-4cec-a1fc-d90cd374462e" containerName="istio-proxy" Jul 24 18:10:45.273101 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.273046 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="20122220-13bf-4cec-a1fc-d90cd374462e" containerName="istio-proxy" Jul 24 18:10:45.273101 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.273072 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20122220-13bf-4cec-a1fc-d90cd374462e" containerName="istio-proxy" Jul 24 18:10:45.273101 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.273080 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="20122220-13bf-4cec-a1fc-d90cd374462e" containerName="istio-proxy" Jul 24 18:10:45.273258 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.273153 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="20122220-13bf-4cec-a1fc-d90cd374462e" containerName="istio-proxy" Jul 24 18:10:45.273258 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.273163 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="20122220-13bf-4cec-a1fc-d90cd374462e" containerName="istio-proxy" Jul 24 18:10:45.275898 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.275881 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8" Jul 24 18:10:45.278498 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.278478 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\"" Jul 24 18:10:45.279029 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.279013 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 18:10:45.300369 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.300344 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8"] Jul 24 18:10:45.366197 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.366163 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de37bc2a-5966-436b-a631-a80b422483c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0858c257-4lbl8\" (UID: \"de37bc2a-5966-436b-a631-a80b422483c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8" Jul 24 18:10:45.467283 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.467250 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de37bc2a-5966-436b-a631-a80b422483c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0858c257-4lbl8\" (UID: \"de37bc2a-5966-436b-a631-a80b422483c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8" Jul 24 18:10:45.467898 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.467882 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de37bc2a-5966-436b-a631-a80b422483c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0858c257-4lbl8\" (UID: \"de37bc2a-5966-436b-a631-a80b422483c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8" Jul 24 18:10:45.585677 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.585609 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8" Jul 24 18:10:45.736677 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:45.736651 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8"] Jul 24 18:10:45.739243 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:10:45.739217 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde37bc2a_5966_436b_a631_a80b422483c3.slice/crio-bf88537782b76750c20d4d1dfdccbf498cd059f00835f0b7a69604f1d51af53b WatchSource:0}: Error finding container bf88537782b76750c20d4d1dfdccbf498cd059f00835f0b7a69604f1d51af53b: Status 404 returned error can't find the container with id bf88537782b76750c20d4d1dfdccbf498cd059f00835f0b7a69604f1d51af53b Jul 24 18:10:46.132199 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:46.132159 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8" event={"ID":"de37bc2a-5966-436b-a631-a80b422483c3","Type":"ContainerStarted","Data":"f9d23ba57102c00126b6c37d6594fb560da07e5cdef586558bdbc276d5d67a61"} Jul 24 18:10:46.132358 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:46.132215 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8" event={"ID":"de37bc2a-5966-436b-a631-a80b422483c3","Type":"ContainerStarted","Data":"bf88537782b76750c20d4d1dfdccbf498cd059f00835f0b7a69604f1d51af53b"} Jul 24 18:10:46.152316 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:46.152107 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8" podStartSLOduration=1.152089826 podStartE2EDuration="1.152089826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:10:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:10:46.150095302 +0000 UTC m=+1841.763101167" watchObservedRunningTime="2026-07-24 18:10:46.152089826 +0000 UTC m=+1841.765095693" Jul 24 18:10:47.137762 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:47.137731 2561 generic.go:356] "Generic (PLEG): container finished" podID="de37bc2a-5966-436b-a631-a80b422483c3" containerID="f9d23ba57102c00126b6c37d6594fb560da07e5cdef586558bdbc276d5d67a61" exitCode=0 Jul 24 18:10:47.138153 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:47.137790 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8" event={"ID":"de37bc2a-5966-436b-a631-a80b422483c3","Type":"ContainerDied","Data":"f9d23ba57102c00126b6c37d6594fb560da07e5cdef586558bdbc276d5d67a61"} Jul 24 18:10:48.266763 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:48.266743 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8" Jul 24 18:10:48.392427 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:48.392355 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de37bc2a-5966-436b-a631-a80b422483c3-maas-api-service-ca\") pod \"de37bc2a-5966-436b-a631-a80b422483c3\" (UID: \"de37bc2a-5966-436b-a631-a80b422483c3\") " Jul 24 18:10:48.392721 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:48.392698 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de37bc2a-5966-436b-a631-a80b422483c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "de37bc2a-5966-436b-a631-a80b422483c3" (UID: "de37bc2a-5966-436b-a631-a80b422483c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:10:48.493753 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:48.493722 2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de37bc2a-5966-436b-a631-a80b422483c3-maas-api-service-ca\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:10:49.147876 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:49.147847 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8" Jul 24 18:10:49.147876 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:49.147864 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8" event={"ID":"de37bc2a-5966-436b-a631-a80b422483c3","Type":"ContainerDied","Data":"bf88537782b76750c20d4d1dfdccbf498cd059f00835f0b7a69604f1d51af53b"} Jul 24 18:10:49.148065 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:49.147915 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf88537782b76750c20d4d1dfdccbf498cd059f00835f0b7a69604f1d51af53b" Jul 24 18:10:55.862325 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:55.862292 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:55.964263 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:55.964227 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:10:56.067136 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:10:56.067107 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:00.329795 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:00.329761 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8"] Jul 24 18:11:00.331845 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:00.331820 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0858c257-4lbl8"] Jul 24 18:11:00.332271 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:00.332249 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:00.940523 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:00.940490 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de37bc2a-5966-436b-a631-a80b422483c3" path="/var/lib/kubelet/pods/de37bc2a-5966-436b-a631-a80b422483c3/volumes" Jul 24 18:11:01.556581 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:01.556543 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl"] Jul 24 18:11:01.556970 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:01.556748 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" podUID="fe54c62e-8695-4e50-8373-a4bcc09f1d81" containerName="istio-proxy" containerID="cri-o://2e994e52612f5088a1f63d7d00342867f4ec5c39bbdebd22ba0a1b390ae1293a" gracePeriod=30 Jul 24 18:11:01.571558 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:01.571527 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:01.798311 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:01.798279 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:02.023949 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.023907 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:02.348196 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.348119 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h"] Jul 24 18:11:02.348461 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.348449 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de37bc2a-5966-436b-a631-a80b422483c3" containerName="revoke-keys" Jul 24 18:11:02.348502 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.348462 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="de37bc2a-5966-436b-a631-a80b422483c3" containerName="revoke-keys" Jul 24 18:11:02.348535 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.348525 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="de37bc2a-5966-436b-a631-a80b422483c3" containerName="revoke-keys" Jul 24 18:11:02.351862 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.351834 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.354497 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.354476 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-eb6423d9-openshift-default-dockercfg-qr749\"" Jul 24 18:11:02.373624 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.373597 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h"] Jul 24 18:11:02.405268 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.405244 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-workload-socket\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.405451 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.405429 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-envoy\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.405577 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.405561 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-data\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.405736 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.405707 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-credential-socket\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.405852 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.405756 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8xzl\" (UniqueName: \"kubernetes.io/projected/d7cc2f08-bada-4605-93b7-476f0d45a48a-kube-api-access-f8xzl\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.405852 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.405807 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-podinfo\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.405852 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.405829 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7cc2f08-bada-4605-93b7-476f0d45a48a-istiod-ca-cert\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.405852 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.405851 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-workload-certs\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.406088 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.405870 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-token\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.411745 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.411720 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:02.507289 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.507258 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8xzl\" (UniqueName: \"kubernetes.io/projected/d7cc2f08-bada-4605-93b7-476f0d45a48a-kube-api-access-f8xzl\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.507451 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.507307 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-podinfo\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.507451 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.507418 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7cc2f08-bada-4605-93b7-476f0d45a48a-istiod-ca-cert\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.507554 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.507453 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-workload-certs\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.507554 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.507475 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-token\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.507710 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.507552 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-workload-socket\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.507710 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.507586 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-envoy\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.507710 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.507634 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-data\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.507862 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.507713 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-credential-socket\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.508035 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.508007 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-workload-socket\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.508209 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.508183 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7cc2f08-bada-4605-93b7-476f0d45a48a-istiod-ca-cert\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.508313 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.508220 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-credential-socket\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.508313 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.508285 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-data\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.508419 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.508321 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-workload-certs\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.509852 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.509824 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-podinfo\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.509976 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.509954 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-envoy\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.515145 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.515117 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8xzl\" (UniqueName: \"kubernetes.io/projected/d7cc2f08-bada-4605-93b7-476f0d45a48a-kube-api-access-f8xzl\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.515417 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.515399 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-token\") pod \"e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.663356 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.663266 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:02.796743 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.796633 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h"] Jul 24 18:11:02.799492 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:11:02.799464 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7cc2f08_bada_4605_93b7_476f0d45a48a.slice/crio-0cd165d85f5ec8386700c1318fd906607582c1c0db16d68ed2da185e435f1659 WatchSource:0}: Error finding container 0cd165d85f5ec8386700c1318fd906607582c1c0db16d68ed2da185e435f1659: Status 404 returned error can't find the container with id 0cd165d85f5ec8386700c1318fd906607582c1c0db16d68ed2da185e435f1659 Jul 24 18:11:02.801650 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.801616 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:11:02.801732 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.801683 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:11:02.801732 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:02.801713 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:11:03.209283 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:03.209239 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" event={"ID":"d7cc2f08-bada-4605-93b7-476f0d45a48a","Type":"ContainerStarted","Data":"1fbea501cc9e86af6256bb0793e5fecba5626a446d7d36c5f154d3d353e5a0ba"} Jul 24 18:11:03.209469 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:03.209293 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" event={"ID":"d7cc2f08-bada-4605-93b7-476f0d45a48a","Type":"ContainerStarted","Data":"0cd165d85f5ec8386700c1318fd906607582c1c0db16d68ed2da185e435f1659"} Jul 24 18:11:03.245288 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:03.245256 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:03.245442 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:03.245393 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" podStartSLOduration=1.245373791 podStartE2EDuration="1.245373791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:11:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:11:03.2434544 +0000 UTC m=+1858.856460305" watchObservedRunningTime="2026-07-24 18:11:03.245373791 +0000 UTC m=+1858.858379657" Jul 24 18:11:03.664392 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:03.664293 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:03.669219 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:03.669193 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:04.214620 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:04.214594 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:04.215423 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:04.215407 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:04.670137 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:04.670055 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:04.796443 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:04.796413 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:04.910469 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:04.910434 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:05.031251 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:05.031221 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:05.500739 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:05.500708 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:05.628062 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:05.628034 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:06.800366 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.800344 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:11:06.846192 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.846125 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istiod-ca-cert\") pod \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " Jul 24 18:11:06.846192 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.846169 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-token\") pod \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " Jul 24 18:11:06.846387 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.846210 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-workload-certs\") pod \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " Jul 24 18:11:06.846387 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.846243 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-envoy\") pod \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " Jul 24 18:11:06.846387 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.846260 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-workload-socket\") pod \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " Jul 24 18:11:06.846387 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.846280 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-credential-socket\") pod \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " Jul 24 18:11:06.846387 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.846299 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnnw2\" (UniqueName: \"kubernetes.io/projected/fe54c62e-8695-4e50-8373-a4bcc09f1d81-kube-api-access-mnnw2\") pod \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " Jul 24 18:11:06.846387 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.846323 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-podinfo\") pod \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " Jul 24 18:11:06.846387 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.846351 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-data\") pod \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\" (UID: \"fe54c62e-8695-4e50-8373-a4bcc09f1d81\") " Jul 24 18:11:06.846715 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.846570 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fe54c62e-8695-4e50-8373-a4bcc09f1d81" (UID: "fe54c62e-8695-4e50-8373-a4bcc09f1d81"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:11:06.846715 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.846581 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fe54c62e-8695-4e50-8373-a4bcc09f1d81" (UID: "fe54c62e-8695-4e50-8373-a4bcc09f1d81"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:11:06.846715 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.846595 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fe54c62e-8695-4e50-8373-a4bcc09f1d81" (UID: "fe54c62e-8695-4e50-8373-a4bcc09f1d81"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:11:06.846715 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.846623 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fe54c62e-8695-4e50-8373-a4bcc09f1d81" (UID: "fe54c62e-8695-4e50-8373-a4bcc09f1d81"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:11:06.847377 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.847357 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-data" (OuterVolumeSpecName: "istio-data") pod "fe54c62e-8695-4e50-8373-a4bcc09f1d81" (UID: "fe54c62e-8695-4e50-8373-a4bcc09f1d81"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:11:06.848593 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.848567 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe54c62e-8695-4e50-8373-a4bcc09f1d81-kube-api-access-mnnw2" (OuterVolumeSpecName: "kube-api-access-mnnw2") pod "fe54c62e-8695-4e50-8373-a4bcc09f1d81" (UID: "fe54c62e-8695-4e50-8373-a4bcc09f1d81"). InnerVolumeSpecName "kube-api-access-mnnw2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:11:06.848674 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.848577 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fe54c62e-8695-4e50-8373-a4bcc09f1d81" (UID: "fe54c62e-8695-4e50-8373-a4bcc09f1d81"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:11:06.848853 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.848828 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fe54c62e-8695-4e50-8373-a4bcc09f1d81" (UID: "fe54c62e-8695-4e50-8373-a4bcc09f1d81"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:11:06.848955 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.848830 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-token" (OuterVolumeSpecName: "istio-token") pod "fe54c62e-8695-4e50-8373-a4bcc09f1d81" (UID: "fe54c62e-8695-4e50-8373-a4bcc09f1d81"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:11:06.948134 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.947869 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:06.948134 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.947890 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:06.948134 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.947898 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:06.948134 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.947907 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:06.948134 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.947920 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:06.948134 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.947933 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:06.948134 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.947955 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnnw2\" (UniqueName: \"kubernetes.io/projected/fe54c62e-8695-4e50-8373-a4bcc09f1d81-kube-api-access-mnnw2\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:06.948134 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.947964 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:06.948134 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:06.947971 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe54c62e-8695-4e50-8373-a4bcc09f1d81-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:07.231164 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:07.231135 2561 generic.go:356] "Generic (PLEG): container finished" podID="fe54c62e-8695-4e50-8373-a4bcc09f1d81" containerID="2e994e52612f5088a1f63d7d00342867f4ec5c39bbdebd22ba0a1b390ae1293a" exitCode=0 Jul 24 18:11:07.231313 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:07.231175 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" event={"ID":"fe54c62e-8695-4e50-8373-a4bcc09f1d81","Type":"ContainerDied","Data":"2e994e52612f5088a1f63d7d00342867f4ec5c39bbdebd22ba0a1b390ae1293a"} Jul 24 18:11:07.231313 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:07.231185 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" Jul 24 18:11:07.231313 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:07.231211 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl" event={"ID":"fe54c62e-8695-4e50-8373-a4bcc09f1d81","Type":"ContainerDied","Data":"62419094a28bfc8939f95719fb2b284006e6fd557f63bb1f1194e8347c46f727"} Jul 24 18:11:07.231313 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:07.231234 2561 scope.go:117] "RemoveContainer" containerID="2e994e52612f5088a1f63d7d00342867f4ec5c39bbdebd22ba0a1b390ae1293a" Jul 24 18:11:07.239690 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:07.239672 2561 scope.go:117] "RemoveContainer" containerID="44414eabbae3f6335322ad338fb26a15d869abf940eebebcff2b7ee0cbe9c588" Jul 24 18:11:07.247568 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:07.247543 2561 scope.go:117] "RemoveContainer" containerID="2e994e52612f5088a1f63d7d00342867f4ec5c39bbdebd22ba0a1b390ae1293a" Jul 24 18:11:07.247802 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:11:07.247783 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e994e52612f5088a1f63d7d00342867f4ec5c39bbdebd22ba0a1b390ae1293a\": container with ID starting with 2e994e52612f5088a1f63d7d00342867f4ec5c39bbdebd22ba0a1b390ae1293a not found: ID does not exist" containerID="2e994e52612f5088a1f63d7d00342867f4ec5c39bbdebd22ba0a1b390ae1293a" Jul 24 18:11:07.247855 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:07.247811 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e994e52612f5088a1f63d7d00342867f4ec5c39bbdebd22ba0a1b390ae1293a"} err="failed to get container status \"2e994e52612f5088a1f63d7d00342867f4ec5c39bbdebd22ba0a1b390ae1293a\": rpc error: code = NotFound desc = could not find container \"2e994e52612f5088a1f63d7d00342867f4ec5c39bbdebd22ba0a1b390ae1293a\": container with ID starting with 2e994e52612f5088a1f63d7d00342867f4ec5c39bbdebd22ba0a1b390ae1293a not found: ID does not exist" Jul 24 18:11:07.247855 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:07.247828 2561 scope.go:117] "RemoveContainer" containerID="44414eabbae3f6335322ad338fb26a15d869abf940eebebcff2b7ee0cbe9c588" Jul 24 18:11:07.247979 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:07.247865 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl"] Jul 24 18:11:07.248056 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:11:07.248040 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44414eabbae3f6335322ad338fb26a15d869abf940eebebcff2b7ee0cbe9c588\": container with ID starting with 44414eabbae3f6335322ad338fb26a15d869abf940eebebcff2b7ee0cbe9c588 not found: ID does not exist" containerID="44414eabbae3f6335322ad338fb26a15d869abf940eebebcff2b7ee0cbe9c588" Jul 24 18:11:07.248094 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:07.248062 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44414eabbae3f6335322ad338fb26a15d869abf940eebebcff2b7ee0cbe9c588"} err="failed to get container status \"44414eabbae3f6335322ad338fb26a15d869abf940eebebcff2b7ee0cbe9c588\": rpc error: code = NotFound desc = could not find container \"44414eabbae3f6335322ad338fb26a15d869abf940eebebcff2b7ee0cbe9c588\": container with ID starting with 44414eabbae3f6335322ad338fb26a15d869abf940eebebcff2b7ee0cbe9c588 not found: ID does not exist" Jul 24 18:11:07.251479 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:07.251459 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0858c257-openshift-default-8674fdbf9b-pcvbl"] Jul 24 18:11:08.939789 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:08.939758 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe54c62e-8695-4e50-8373-a4bcc09f1d81" path="/var/lib/kubelet/pods/fe54c62e-8695-4e50-8373-a4bcc09f1d81/volumes" Jul 24 18:11:18.277212 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:18.277180 2561 generic.go:356] "Generic (PLEG): container finished" podID="d7cc2f08-bada-4605-93b7-476f0d45a48a" containerID="1fbea501cc9e86af6256bb0793e5fecba5626a446d7d36c5f154d3d353e5a0ba" exitCode=0 Jul 24 18:11:18.277634 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:18.277238 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" event={"ID":"d7cc2f08-bada-4605-93b7-476f0d45a48a","Type":"ContainerDied","Data":"1fbea501cc9e86af6256bb0793e5fecba5626a446d7d36c5f154d3d353e5a0ba"} Jul 24 18:11:18.277713 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:18.277638 2561 scope.go:117] "RemoveContainer" containerID="1fbea501cc9e86af6256bb0793e5fecba5626a446d7d36c5f154d3d353e5a0ba" Jul 24 18:11:18.278793 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:18.278753 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:11:18.278906 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:18.278843 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:11:18.278906 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:18.278891 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:11:19.215035 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:19.214997 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:19.285857 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:19.285813 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" event={"ID":"d7cc2f08-bada-4605-93b7-476f0d45a48a","Type":"ContainerStarted","Data":"c37e01dbf7f86ac872b8f8c2e002fc217ec55d5fd60456075921b4d76df53c0e"} Jul 24 18:11:19.663510 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:19.663433 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:19.664559 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:19.664528 2561 patch_prober.go:28] interesting pod/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 24 18:11:19.664700 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:19.664594 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" podUID="d7cc2f08-bada-4605-93b7-476f0d45a48a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 24 18:11:20.664038 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:20.664003 2561 patch_prober.go:28] interesting pod/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 24 18:11:20.664433 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:20.664058 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" podUID="d7cc2f08-bada-4605-93b7-476f0d45a48a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 24 18:11:21.664101 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:21.664064 2561 patch_prober.go:28] interesting pod/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 24 18:11:21.664572 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:21.664165 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" podUID="d7cc2f08-bada-4605-93b7-476f0d45a48a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 24 18:11:22.664466 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:22.664430 2561 patch_prober.go:28] interesting pod/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 24 18:11:22.664861 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:22.664493 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" podUID="d7cc2f08-bada-4605-93b7-476f0d45a48a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 24 18:11:23.668346 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:23.668312 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:23.668729 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:23.668567 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:23.669258 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:23.669236 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:29.561449 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:29.561375 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr"] Jul 24 18:11:29.561797 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:29.561657 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe54c62e-8695-4e50-8373-a4bcc09f1d81" containerName="istio-proxy" Jul 24 18:11:29.561797 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:29.561668 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe54c62e-8695-4e50-8373-a4bcc09f1d81" containerName="istio-proxy" Jul 24 18:11:29.561797 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:29.561678 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe54c62e-8695-4e50-8373-a4bcc09f1d81" containerName="istio-proxy" Jul 24 18:11:29.561797 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:29.561684 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe54c62e-8695-4e50-8373-a4bcc09f1d81" containerName="istio-proxy" Jul 24 18:11:29.561797 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:29.561743 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe54c62e-8695-4e50-8373-a4bcc09f1d81" containerName="istio-proxy" Jul 24 18:11:29.561797 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:29.561750 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe54c62e-8695-4e50-8373-a4bcc09f1d81" containerName="istio-proxy" Jul 24 18:11:29.564730 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:29.564715 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr" Jul 24 18:11:29.567047 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:29.567023 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 18:11:29.567176 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:29.567160 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\"" Jul 24 18:11:29.573591 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:29.573566 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr"] Jul 24 18:11:29.656805 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:29.656776 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ea68178-7da0-4e4c-915b-4edac834c0e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr\" (UID: \"5ea68178-7da0-4e4c-915b-4edac834c0e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr" Jul 24 18:11:29.757828 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:29.757783 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ea68178-7da0-4e4c-915b-4edac834c0e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr\" (UID: \"5ea68178-7da0-4e4c-915b-4edac834c0e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr" Jul 24 18:11:29.758462 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:29.758439 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ea68178-7da0-4e4c-915b-4edac834c0e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr\" (UID: \"5ea68178-7da0-4e4c-915b-4edac834c0e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr" Jul 24 18:11:29.876308 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:29.876231 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr" Jul 24 18:11:30.003780 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:30.003751 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr"] Jul 24 18:11:30.006446 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:11:30.006415 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ea68178_7da0_4e4c_915b_4edac834c0e0.slice/crio-9e80d3285b581eda910ba940046aabdf496f279639d32dd49dda25cfa20bbdf0 WatchSource:0}: Error finding container 9e80d3285b581eda910ba940046aabdf496f279639d32dd49dda25cfa20bbdf0: Status 404 returned error can't find the container with id 9e80d3285b581eda910ba940046aabdf496f279639d32dd49dda25cfa20bbdf0 Jul 24 18:11:30.335687 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:30.335653 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr" event={"ID":"5ea68178-7da0-4e4c-915b-4edac834c0e0","Type":"ContainerStarted","Data":"6dd9a4269198682634a0d89440992199d1fad1f84537aed154a06aebee870373"} Jul 24 18:11:30.335839 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:30.335709 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr" event={"ID":"5ea68178-7da0-4e4c-915b-4edac834c0e0","Type":"ContainerStarted","Data":"9e80d3285b581eda910ba940046aabdf496f279639d32dd49dda25cfa20bbdf0"} Jul 24 18:11:30.365022 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:30.364978 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr" podStartSLOduration=1.364963036 podStartE2EDuration="1.364963036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:11:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:11:30.361448216 +0000 UTC m=+1885.974454083" watchObservedRunningTime="2026-07-24 18:11:30.364963036 +0000 UTC m=+1885.977968902" Jul 24 18:11:31.341087 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:31.341049 2561 generic.go:356] "Generic (PLEG): container finished" podID="5ea68178-7da0-4e4c-915b-4edac834c0e0" containerID="6dd9a4269198682634a0d89440992199d1fad1f84537aed154a06aebee870373" exitCode=0 Jul 24 18:11:31.341451 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:31.341137 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr" event={"ID":"5ea68178-7da0-4e4c-915b-4edac834c0e0","Type":"ContainerDied","Data":"6dd9a4269198682634a0d89440992199d1fad1f84537aed154a06aebee870373"} Jul 24 18:11:32.467376 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:32.467351 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr" Jul 24 18:11:32.583109 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:32.583077 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ea68178-7da0-4e4c-915b-4edac834c0e0-maas-api-service-ca\") pod \"5ea68178-7da0-4e4c-915b-4edac834c0e0\" (UID: \"5ea68178-7da0-4e4c-915b-4edac834c0e0\") " Jul 24 18:11:32.583431 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:32.583408 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ea68178-7da0-4e4c-915b-4edac834c0e0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5ea68178-7da0-4e4c-915b-4edac834c0e0" (UID: "5ea68178-7da0-4e4c-915b-4edac834c0e0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:11:32.684147 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:32.684117 2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ea68178-7da0-4e4c-915b-4edac834c0e0-maas-api-service-ca\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:33.350210 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:33.350134 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr" Jul 24 18:11:33.350210 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:33.350144 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr" event={"ID":"5ea68178-7da0-4e4c-915b-4edac834c0e0","Type":"ContainerDied","Data":"9e80d3285b581eda910ba940046aabdf496f279639d32dd49dda25cfa20bbdf0"} Jul 24 18:11:33.350210 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:33.350185 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e80d3285b581eda910ba940046aabdf496f279639d32dd49dda25cfa20bbdf0" Jul 24 18:11:40.149016 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:40.148982 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:40.249018 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:40.248982 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:40.349073 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:40.349040 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:44.671583 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:44.671524 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:44.681194 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:44.681162 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr"] Jul 24 18:11:44.682769 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:44.682749 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb6423d9-8zjgr"] Jul 24 18:11:44.825774 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:44.825741 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h"] Jul 24 18:11:44.826023 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:44.825975 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" podUID="d7cc2f08-bada-4605-93b7-476f0d45a48a" containerName="istio-proxy" containerID="cri-o://c37e01dbf7f86ac872b8f8c2e002fc217ec55d5fd60456075921b4d76df53c0e" gracePeriod=30 Jul 24 18:11:44.849092 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:44.849062 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:44.941024 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:44.940996 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ea68178-7da0-4e4c-915b-4edac834c0e0" path="/var/lib/kubelet/pods/5ea68178-7da0-4e4c-915b-4edac834c0e0/volumes" Jul 24 18:11:45.092348 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:45.092316 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:45.626655 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:45.626622 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:45.733091 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:45.733054 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:45.825953 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:45.825913 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:47.354927 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:47.354889 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:50.078151 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.078126 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:50.130091 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.129991 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-token\") pod \"d7cc2f08-bada-4605-93b7-476f0d45a48a\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " Jul 24 18:11:50.130091 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.130051 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7cc2f08-bada-4605-93b7-476f0d45a48a-istiod-ca-cert\") pod \"d7cc2f08-bada-4605-93b7-476f0d45a48a\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " Jul 24 18:11:50.130091 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.130081 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-envoy\") pod \"d7cc2f08-bada-4605-93b7-476f0d45a48a\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " Jul 24 18:11:50.130367 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.130129 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-workload-socket\") pod \"d7cc2f08-bada-4605-93b7-476f0d45a48a\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " Jul 24 18:11:50.130367 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.130149 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-data\") pod \"d7cc2f08-bada-4605-93b7-476f0d45a48a\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " Jul 24 18:11:50.130367 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.130170 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-podinfo\") pod \"d7cc2f08-bada-4605-93b7-476f0d45a48a\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " Jul 24 18:11:50.130367 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.130198 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8xzl\" (UniqueName: \"kubernetes.io/projected/d7cc2f08-bada-4605-93b7-476f0d45a48a-kube-api-access-f8xzl\") pod \"d7cc2f08-bada-4605-93b7-476f0d45a48a\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " Jul 24 18:11:50.130367 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.130262 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-workload-certs\") pod \"d7cc2f08-bada-4605-93b7-476f0d45a48a\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " Jul 24 18:11:50.130367 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.130287 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-credential-socket\") pod \"d7cc2f08-bada-4605-93b7-476f0d45a48a\" (UID: \"d7cc2f08-bada-4605-93b7-476f0d45a48a\") " Jul 24 18:11:50.130669 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.130453 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7cc2f08-bada-4605-93b7-476f0d45a48a" (UID: "d7cc2f08-bada-4605-93b7-476f0d45a48a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:11:50.130669 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.130538 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7cc2f08-bada-4605-93b7-476f0d45a48a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7cc2f08-bada-4605-93b7-476f0d45a48a" (UID: "d7cc2f08-bada-4605-93b7-476f0d45a48a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:11:50.130786 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.130677 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7cc2f08-bada-4605-93b7-476f0d45a48a" (UID: "d7cc2f08-bada-4605-93b7-476f0d45a48a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:11:50.130786 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.130708 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7cc2f08-bada-4605-93b7-476f0d45a48a-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:50.130786 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.130730 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:50.130960 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.130804 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7cc2f08-bada-4605-93b7-476f0d45a48a" (UID: "d7cc2f08-bada-4605-93b7-476f0d45a48a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:11:50.131208 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.131176 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7cc2f08-bada-4605-93b7-476f0d45a48a" (UID: "d7cc2f08-bada-4605-93b7-476f0d45a48a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:11:50.132338 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.132310 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7cc2f08-bada-4605-93b7-476f0d45a48a" (UID: "d7cc2f08-bada-4605-93b7-476f0d45a48a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:11:50.132338 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.132327 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7cc2f08-bada-4605-93b7-476f0d45a48a" (UID: "d7cc2f08-bada-4605-93b7-476f0d45a48a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:11:50.132478 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.132444 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7cc2f08-bada-4605-93b7-476f0d45a48a" (UID: "d7cc2f08-bada-4605-93b7-476f0d45a48a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:11:50.132515 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.132500 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7cc2f08-bada-4605-93b7-476f0d45a48a-kube-api-access-f8xzl" (OuterVolumeSpecName: "kube-api-access-f8xzl") pod "d7cc2f08-bada-4605-93b7-476f0d45a48a" (UID: "d7cc2f08-bada-4605-93b7-476f0d45a48a"). InnerVolumeSpecName "kube-api-access-f8xzl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:11:50.231743 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.231708 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:50.231743 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.231739 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:50.231743 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.231752 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:50.232002 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.231766 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:50.232002 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.231778 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:50.232002 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.231785 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7cc2f08-bada-4605-93b7-476f0d45a48a-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:50.232002 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.231793 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8xzl\" (UniqueName: \"kubernetes.io/projected/d7cc2f08-bada-4605-93b7-476f0d45a48a-kube-api-access-f8xzl\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:11:50.423177 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.423089 2561 generic.go:356] "Generic (PLEG): container finished" podID="d7cc2f08-bada-4605-93b7-476f0d45a48a" containerID="c37e01dbf7f86ac872b8f8c2e002fc217ec55d5fd60456075921b4d76df53c0e" exitCode=0 Jul 24 18:11:50.423177 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.423141 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" Jul 24 18:11:50.423344 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.423169 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" event={"ID":"d7cc2f08-bada-4605-93b7-476f0d45a48a","Type":"ContainerDied","Data":"c37e01dbf7f86ac872b8f8c2e002fc217ec55d5fd60456075921b4d76df53c0e"} Jul 24 18:11:50.423344 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.423224 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h" event={"ID":"d7cc2f08-bada-4605-93b7-476f0d45a48a","Type":"ContainerDied","Data":"0cd165d85f5ec8386700c1318fd906607582c1c0db16d68ed2da185e435f1659"} Jul 24 18:11:50.423344 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.423246 2561 scope.go:117] "RemoveContainer" containerID="c37e01dbf7f86ac872b8f8c2e002fc217ec55d5fd60456075921b4d76df53c0e" Jul 24 18:11:50.432533 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.432513 2561 scope.go:117] "RemoveContainer" containerID="1fbea501cc9e86af6256bb0793e5fecba5626a446d7d36c5f154d3d353e5a0ba" Jul 24 18:11:50.440371 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.440347 2561 scope.go:117] "RemoveContainer" containerID="c37e01dbf7f86ac872b8f8c2e002fc217ec55d5fd60456075921b4d76df53c0e" Jul 24 18:11:50.440643 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:11:50.440625 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c37e01dbf7f86ac872b8f8c2e002fc217ec55d5fd60456075921b4d76df53c0e\": container with ID starting with c37e01dbf7f86ac872b8f8c2e002fc217ec55d5fd60456075921b4d76df53c0e not found: ID does not exist" containerID="c37e01dbf7f86ac872b8f8c2e002fc217ec55d5fd60456075921b4d76df53c0e" Jul 24 18:11:50.440688 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.440652 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c37e01dbf7f86ac872b8f8c2e002fc217ec55d5fd60456075921b4d76df53c0e"} err="failed to get container status \"c37e01dbf7f86ac872b8f8c2e002fc217ec55d5fd60456075921b4d76df53c0e\": rpc error: code = NotFound desc = could not find container \"c37e01dbf7f86ac872b8f8c2e002fc217ec55d5fd60456075921b4d76df53c0e\": container with ID starting with c37e01dbf7f86ac872b8f8c2e002fc217ec55d5fd60456075921b4d76df53c0e not found: ID does not exist" Jul 24 18:11:50.440688 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.440670 2561 scope.go:117] "RemoveContainer" containerID="1fbea501cc9e86af6256bb0793e5fecba5626a446d7d36c5f154d3d353e5a0ba" Jul 24 18:11:50.440923 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:11:50.440907 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fbea501cc9e86af6256bb0793e5fecba5626a446d7d36c5f154d3d353e5a0ba\": container with ID starting with 1fbea501cc9e86af6256bb0793e5fecba5626a446d7d36c5f154d3d353e5a0ba not found: ID does not exist" containerID="1fbea501cc9e86af6256bb0793e5fecba5626a446d7d36c5f154d3d353e5a0ba" Jul 24 18:11:50.440982 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.440927 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fbea501cc9e86af6256bb0793e5fecba5626a446d7d36c5f154d3d353e5a0ba"} err="failed to get container status \"1fbea501cc9e86af6256bb0793e5fecba5626a446d7d36c5f154d3d353e5a0ba\": rpc error: code = NotFound desc = could not find container \"1fbea501cc9e86af6256bb0793e5fecba5626a446d7d36c5f154d3d353e5a0ba\": container with ID starting with 1fbea501cc9e86af6256bb0793e5fecba5626a446d7d36c5f154d3d353e5a0ba not found: ID does not exist" Jul 24 18:11:50.445877 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.445857 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h"] Jul 24 18:11:50.449879 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.449860 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-eb6423d9-openshift-default-5b67f75fc6-qwf6h"] Jul 24 18:11:50.940142 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:50.940099 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7cc2f08-bada-4605-93b7-476f0d45a48a" path="/var/lib/kubelet/pods/d7cc2f08-bada-4605-93b7-476f0d45a48a/volumes" Jul 24 18:11:53.249879 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:53.249846 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:53.359277 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:53.359245 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:53.458700 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:53.458667 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:53.713232 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:53.713198 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:11:54.067874 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:11:54.067793 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:12:20.122079 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:12:20.122042 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:12:27.065289 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:12:27.065252 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:12:27.173680 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:12:27.173648 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:12:27.267346 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:12:27.267286 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:12:43.194840 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:12:43.194801 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:12:49.609262 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:12:49.609222 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:12:49.720491 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:12:49.720462 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:12:49.833234 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:12:49.833204 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:12:59.578434 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:12:59.578349 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:00.395003 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:00.394967 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:01.205857 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:01.205821 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:07.801598 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:07.801563 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:07.916198 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:07.916158 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:08.034292 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:08.034261 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:08.139278 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:08.139198 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:08.945403 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:08.945371 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:09.052762 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:09.052729 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:09.148217 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:09.148180 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:10.838906 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:10.838871 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:16.881673 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:16.881639 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:16.980330 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:16.980297 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:17.577279 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:17.577241 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:23.547769 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:23.547734 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:23.650608 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:23.650545 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:23.812600 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:23.812512 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:24.135622 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.135539 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w"] Jul 24 18:13:24.135918 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.135899 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea68178-7da0-4e4c-915b-4edac834c0e0" containerName="revoke-keys" Jul 24 18:13:24.136057 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.135921 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea68178-7da0-4e4c-915b-4edac834c0e0" containerName="revoke-keys" Jul 24 18:13:24.136057 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.135954 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7cc2f08-bada-4605-93b7-476f0d45a48a" containerName="istio-proxy" Jul 24 18:13:24.136057 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.135963 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7cc2f08-bada-4605-93b7-476f0d45a48a" containerName="istio-proxy" Jul 24 18:13:24.136057 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.135975 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7cc2f08-bada-4605-93b7-476f0d45a48a" containerName="istio-proxy" Jul 24 18:13:24.136057 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.135983 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7cc2f08-bada-4605-93b7-476f0d45a48a" containerName="istio-proxy" Jul 24 18:13:24.136307 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.136086 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7cc2f08-bada-4605-93b7-476f0d45a48a" containerName="istio-proxy" Jul 24 18:13:24.136307 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.136098 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea68178-7da0-4e4c-915b-4edac834c0e0" containerName="revoke-keys" Jul 24 18:13:24.139022 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.138997 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.141641 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.141622 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8e8a9ad3-openshift-default-dockercfg-xc4vs\"" Jul 24 18:13:24.173986 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.173925 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w"] Jul 24 18:13:24.185333 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.185297 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:24.257092 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.257048 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-workload-socket\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.257308 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.257112 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-workload-certs\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.257308 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.257170 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-podinfo\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.257308 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.257285 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-token\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.257499 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.257323 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwkj5\" (UniqueName: \"kubernetes.io/projected/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-kube-api-access-xwkj5\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.257499 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.257405 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-envoy\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.257499 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.257465 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-credential-socket\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.257499 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.257488 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-data\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.257702 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.257520 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istiod-ca-cert\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.358961 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.358895 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-credential-socket\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.358961 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.358964 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-data\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.359193 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.358990 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istiod-ca-cert\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.359193 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.359018 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-workload-socket\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.359193 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.359039 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-workload-certs\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.359193 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.359056 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-podinfo\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.359193 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.359098 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-token\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.359193 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.359125 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwkj5\" (UniqueName: \"kubernetes.io/projected/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-kube-api-access-xwkj5\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.359193 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.359166 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-envoy\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.359545 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.359383 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-credential-socket\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.359545 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.359425 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-data\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.359545 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.359453 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-workload-socket\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.359545 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.359512 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-workload-certs\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.360051 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.360033 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istiod-ca-cert\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.361609 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.361587 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-envoy\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.361726 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.361710 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-podinfo\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.368204 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.368172 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-token\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.368331 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.368302 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwkj5\" (UniqueName: \"kubernetes.io/projected/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-kube-api-access-xwkj5\") pod \"e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.376003 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.375973 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:24.450087 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.450040 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:24.585421 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.585345 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w"] Jul 24 18:13:24.588169 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:13:24.588141 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e7d9b3b_055e_4d3e_bd56_52a11ac843e0.slice/crio-50b953155cc33023f6be4c2697a53fa95d295a222646fef030f44e59456d40b6 WatchSource:0}: Error finding container 50b953155cc33023f6be4c2697a53fa95d295a222646fef030f44e59456d40b6: Status 404 returned error can't find the container with id 50b953155cc33023f6be4c2697a53fa95d295a222646fef030f44e59456d40b6 Jul 24 18:13:24.590240 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.590207 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:13:24.590316 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.590292 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:13:24.590351 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.590323 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:13:24.834820 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.834739 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" event={"ID":"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0","Type":"ContainerStarted","Data":"3a246f666fa2525a6c5239413d272307eaa8510824e084ad421358c99df2c466"} Jul 24 18:13:24.834820 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.834803 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" event={"ID":"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0","Type":"ContainerStarted","Data":"50b953155cc33023f6be4c2697a53fa95d295a222646fef030f44e59456d40b6"} Jul 24 18:13:24.858998 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:24.858915 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" podStartSLOduration=0.858889603 podStartE2EDuration="858.889603ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:13:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:13:24.857594133 +0000 UTC m=+2000.470600020" watchObservedRunningTime="2026-07-24 18:13:24.858889603 +0000 UTC m=+2000.471895470" Jul 24 18:13:25.451201 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:25.451155 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:25.455983 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:25.455928 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:25.838976 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:25.838874 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:25.839709 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:25.839693 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:26.982272 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:26.982238 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:27.122658 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:27.122625 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:27.263881 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:27.263804 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:27.622512 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:27.622432 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:27.859630 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:27.859594 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:27.958727 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:27.958692 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:31.474166 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:31.474120 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:32.301399 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:32.301368 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:32.436766 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:32.436729 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:32.728852 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:32.728819 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:38.762706 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:38.762666 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:38.863203 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:38.863170 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:38.902351 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:38.902314 2561 generic.go:356] "Generic (PLEG): container finished" podID="8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" containerID="3a246f666fa2525a6c5239413d272307eaa8510824e084ad421358c99df2c466" exitCode=0 Jul 24 18:13:38.902536 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:38.902387 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" event={"ID":"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0","Type":"ContainerDied","Data":"3a246f666fa2525a6c5239413d272307eaa8510824e084ad421358c99df2c466"} Jul 24 18:13:38.902782 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:38.902766 2561 scope.go:117] "RemoveContainer" containerID="3a246f666fa2525a6c5239413d272307eaa8510824e084ad421358c99df2c466" Jul 24 18:13:38.903783 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:38.903754 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:13:38.903851 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:38.903826 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:13:38.903905 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:38.903858 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:13:39.147347 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:39.147245 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq"] Jul 24 18:13:39.147612 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:39.147598 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7cc2f08-bada-4605-93b7-476f0d45a48a" containerName="istio-proxy" Jul 24 18:13:39.150359 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:39.150326 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq" Jul 24 18:13:39.152983 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:39.152958 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 18:13:39.153413 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:39.153378 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\"" Jul 24 18:13:39.161594 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:39.161563 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq"] Jul 24 18:13:39.291304 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:39.291258 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab402195-e4b9-4e6b-91e6-b1fcea40cdf7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq\" (UID: \"ab402195-e4b9-4e6b-91e6-b1fcea40cdf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq" Jul 24 18:13:39.392453 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:39.392415 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab402195-e4b9-4e6b-91e6-b1fcea40cdf7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq\" (UID: \"ab402195-e4b9-4e6b-91e6-b1fcea40cdf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq" Jul 24 18:13:39.393062 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:39.393041 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab402195-e4b9-4e6b-91e6-b1fcea40cdf7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq\" (UID: \"ab402195-e4b9-4e6b-91e6-b1fcea40cdf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq" Jul 24 18:13:39.463179 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:39.463143 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq" Jul 24 18:13:39.595158 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:39.595123 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq"] Jul 24 18:13:39.598150 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:13:39.598105 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab402195_e4b9_4e6b_91e6_b1fcea40cdf7.slice/crio-cd250104036a7be812d6af2b6bac5a866fdeb3e213a81e5f205a0975a3eff96d WatchSource:0}: Error finding container cd250104036a7be812d6af2b6bac5a866fdeb3e213a81e5f205a0975a3eff96d: Status 404 returned error can't find the container with id cd250104036a7be812d6af2b6bac5a866fdeb3e213a81e5f205a0975a3eff96d Jul 24 18:13:39.908799 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:39.908763 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" event={"ID":"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0","Type":"ContainerStarted","Data":"fe8bba26468c0b8ae32ef06c9a5ecf915d32c7b18ade7c5618683eaf274851b4"} Jul 24 18:13:39.910345 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:39.910319 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq" event={"ID":"ab402195-e4b9-4e6b-91e6-b1fcea40cdf7","Type":"ContainerStarted","Data":"681721aff2cf01f6b2d723b3c541b3abb0589340f4ea69270649e63dd8a90ea2"} Jul 24 18:13:39.910482 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:39.910367 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq" event={"ID":"ab402195-e4b9-4e6b-91e6-b1fcea40cdf7","Type":"ContainerStarted","Data":"cd250104036a7be812d6af2b6bac5a866fdeb3e213a81e5f205a0975a3eff96d"} Jul 24 18:13:39.949899 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:39.949843 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq" podStartSLOduration=0.949824337 podStartE2EDuration="949.824337ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:13:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:13:39.948549855 +0000 UTC m=+2015.561555723" watchObservedRunningTime="2026-07-24 18:13:39.949824337 +0000 UTC m=+2015.562830203" Jul 24 18:13:40.451113 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:40.451067 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:40.452032 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:40.452011 2561 patch_prober.go:28] interesting pod/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 24 18:13:40.452141 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:40.452052 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" podUID="8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 24 18:13:40.839054 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:40.839016 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:40.915769 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:40.915736 2561 generic.go:356] "Generic (PLEG): container finished" podID="ab402195-e4b9-4e6b-91e6-b1fcea40cdf7" containerID="681721aff2cf01f6b2d723b3c541b3abb0589340f4ea69270649e63dd8a90ea2" exitCode=0 Jul 24 18:13:40.916275 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:40.915825 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq" event={"ID":"ab402195-e4b9-4e6b-91e6-b1fcea40cdf7","Type":"ContainerDied","Data":"681721aff2cf01f6b2d723b3c541b3abb0589340f4ea69270649e63dd8a90ea2"} Jul 24 18:13:41.450749 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:41.450709 2561 patch_prober.go:28] interesting pod/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 24 18:13:41.450913 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:41.450774 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" podUID="8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 24 18:13:42.065774 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:42.065749 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq" Jul 24 18:13:42.219703 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:42.219668 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab402195-e4b9-4e6b-91e6-b1fcea40cdf7-maas-api-service-ca\") pod \"ab402195-e4b9-4e6b-91e6-b1fcea40cdf7\" (UID: \"ab402195-e4b9-4e6b-91e6-b1fcea40cdf7\") " Jul 24 18:13:42.220035 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:42.220012 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab402195-e4b9-4e6b-91e6-b1fcea40cdf7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ab402195-e4b9-4e6b-91e6-b1fcea40cdf7" (UID: "ab402195-e4b9-4e6b-91e6-b1fcea40cdf7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:13:42.321155 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:42.321118 2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab402195-e4b9-4e6b-91e6-b1fcea40cdf7-maas-api-service-ca\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:13:42.450837 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:42.450799 2561 patch_prober.go:28] interesting pod/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 24 18:13:42.451040 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:42.450853 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" podUID="8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 24 18:13:42.925380 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:42.925346 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq" event={"ID":"ab402195-e4b9-4e6b-91e6-b1fcea40cdf7","Type":"ContainerDied","Data":"cd250104036a7be812d6af2b6bac5a866fdeb3e213a81e5f205a0975a3eff96d"} Jul 24 18:13:42.925548 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:42.925426 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd250104036a7be812d6af2b6bac5a866fdeb3e213a81e5f205a0975a3eff96d" Jul 24 18:13:42.925548 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:42.925360 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq" Jul 24 18:13:43.451397 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:43.451359 2561 patch_prober.go:28] interesting pod/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 24 18:13:43.451907 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:43.451421 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" podUID="8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 24 18:13:44.454277 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:44.454239 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:44.455239 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:44.455221 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:13:49.306100 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:49.306068 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:49.406411 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:49.406364 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:49.506812 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:49.506725 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:54.219589 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:54.219549 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq"] Jul 24 18:13:54.224816 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:54.224778 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8e8a9ad3-5k2jq"] Jul 24 18:13:54.236807 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:54.236778 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:54.606155 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:54.606065 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:13:54.939518 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:13:54.939486 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab402195-e4b9-4e6b-91e6-b1fcea40cdf7" path="/var/lib/kubelet/pods/ab402195-e4b9-4e6b-91e6-b1fcea40cdf7/volumes" Jul 24 18:14:00.374696 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:00.374664 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:00.474964 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:00.474907 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:00.635701 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:00.635619 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w"] Jul 24 18:14:00.635862 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:00.635810 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" podUID="8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" containerName="istio-proxy" containerID="cri-o://fe8bba26468c0b8ae32ef06c9a5ecf915d32c7b18ade7c5618683eaf274851b4" gracePeriod=30 Jul 24 18:14:00.696230 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:00.696195 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:00.904011 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:00.903893 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:01.031532 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:01.031499 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:01.131727 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:01.131669 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:01.302388 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:01.302358 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:05.877929 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.877901 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:14:05.925733 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.925654 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-workload-certs\") pod \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " Jul 24 18:14:05.925733 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.925696 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istiod-ca-cert\") pod \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " Jul 24 18:14:05.925733 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.925726 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-credential-socket\") pod \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " Jul 24 18:14:05.925996 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.925870 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-data\") pod \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " Jul 24 18:14:05.925996 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.925963 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-envoy\") pod \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " Jul 24 18:14:05.926086 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.925999 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-podinfo\") pod \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " Jul 24 18:14:05.926086 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.926043 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwkj5\" (UniqueName: \"kubernetes.io/projected/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-kube-api-access-xwkj5\") pod \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " Jul 24 18:14:05.926086 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.926043 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" (UID: "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:14:05.926086 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.926060 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" (UID: "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:14:05.926086 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.926077 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-token\") pod \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " Jul 24 18:14:05.926345 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.926113 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-workload-socket\") pod \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\" (UID: \"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0\") " Jul 24 18:14:05.926345 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.926124 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" (UID: "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:14:05.926462 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.926359 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:14:05.926462 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.926375 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:14:05.926462 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.926387 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:14:05.926462 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.926404 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" (UID: "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:14:05.926786 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.926757 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-data" (OuterVolumeSpecName: "istio-data") pod "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" (UID: "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:14:05.928188 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.928159 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" (UID: "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:14:05.928313 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.928253 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-kube-api-access-xwkj5" (OuterVolumeSpecName: "kube-api-access-xwkj5") pod "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" (UID: "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0"). InnerVolumeSpecName "kube-api-access-xwkj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:14:05.928363 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.928320 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" (UID: "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:14:05.928400 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:05.928386 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-token" (OuterVolumeSpecName: "istio-token") pod "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" (UID: "8e7d9b3b-055e-4d3e-bd56-52a11ac843e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:14:06.024694 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.024653 2561 generic.go:356] "Generic (PLEG): container finished" podID="8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" containerID="fe8bba26468c0b8ae32ef06c9a5ecf915d32c7b18ade7c5618683eaf274851b4" exitCode=0 Jul 24 18:14:06.024899 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.024709 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" Jul 24 18:14:06.024899 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.024741 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" event={"ID":"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0","Type":"ContainerDied","Data":"fe8bba26468c0b8ae32ef06c9a5ecf915d32c7b18ade7c5618683eaf274851b4"} Jul 24 18:14:06.024899 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.024809 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w" event={"ID":"8e7d9b3b-055e-4d3e-bd56-52a11ac843e0","Type":"ContainerDied","Data":"50b953155cc33023f6be4c2697a53fa95d295a222646fef030f44e59456d40b6"} Jul 24 18:14:06.024899 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.024842 2561 scope.go:117] "RemoveContainer" containerID="fe8bba26468c0b8ae32ef06c9a5ecf915d32c7b18ade7c5618683eaf274851b4" Jul 24 18:14:06.026997 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.026964 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:14:06.026997 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.026993 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:14:06.027187 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.027003 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:14:06.027187 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.027012 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:14:06.027187 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.027024 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:14:06.027187 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.027033 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwkj5\" (UniqueName: \"kubernetes.io/projected/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0-kube-api-access-xwkj5\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:14:06.033859 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.033841 2561 scope.go:117] "RemoveContainer" containerID="3a246f666fa2525a6c5239413d272307eaa8510824e084ad421358c99df2c466" Jul 24 18:14:06.042113 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.042092 2561 scope.go:117] "RemoveContainer" containerID="fe8bba26468c0b8ae32ef06c9a5ecf915d32c7b18ade7c5618683eaf274851b4" Jul 24 18:14:06.042376 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:14:06.042354 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe8bba26468c0b8ae32ef06c9a5ecf915d32c7b18ade7c5618683eaf274851b4\": container with ID starting with fe8bba26468c0b8ae32ef06c9a5ecf915d32c7b18ade7c5618683eaf274851b4 not found: ID does not exist" containerID="fe8bba26468c0b8ae32ef06c9a5ecf915d32c7b18ade7c5618683eaf274851b4" Jul 24 18:14:06.042451 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.042389 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe8bba26468c0b8ae32ef06c9a5ecf915d32c7b18ade7c5618683eaf274851b4"} err="failed to get container status \"fe8bba26468c0b8ae32ef06c9a5ecf915d32c7b18ade7c5618683eaf274851b4\": rpc error: code = NotFound desc = could not find container \"fe8bba26468c0b8ae32ef06c9a5ecf915d32c7b18ade7c5618683eaf274851b4\": container with ID starting with fe8bba26468c0b8ae32ef06c9a5ecf915d32c7b18ade7c5618683eaf274851b4 not found: ID does not exist" Jul 24 18:14:06.042451 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.042417 2561 scope.go:117] "RemoveContainer" containerID="3a246f666fa2525a6c5239413d272307eaa8510824e084ad421358c99df2c466" Jul 24 18:14:06.042677 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:14:06.042661 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a246f666fa2525a6c5239413d272307eaa8510824e084ad421358c99df2c466\": container with ID starting with 3a246f666fa2525a6c5239413d272307eaa8510824e084ad421358c99df2c466 not found: ID does not exist" containerID="3a246f666fa2525a6c5239413d272307eaa8510824e084ad421358c99df2c466" Jul 24 18:14:06.042718 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.042685 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a246f666fa2525a6c5239413d272307eaa8510824e084ad421358c99df2c466"} err="failed to get container status \"3a246f666fa2525a6c5239413d272307eaa8510824e084ad421358c99df2c466\": rpc error: code = NotFound desc = could not find container \"3a246f666fa2525a6c5239413d272307eaa8510824e084ad421358c99df2c466\": container with ID starting with 3a246f666fa2525a6c5239413d272307eaa8510824e084ad421358c99df2c466 not found: ID does not exist" Jul 24 18:14:06.046111 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.046087 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w"] Jul 24 18:14:06.050320 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.050296 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8e8a9ad3-openshift-default-5f5d7bfccb-7sh8w"] Jul 24 18:14:06.940068 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:06.940036 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" path="/var/lib/kubelet/pods/8e7d9b3b-055e-4d3e-bd56-52a11ac843e0/volumes" Jul 24 18:14:07.086377 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:07.086346 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:07.209151 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:07.209064 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:07.369636 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:07.369608 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:07.544272 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:07.544161 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:13.283069 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:13.283032 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:13.379618 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:13.379552 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:20.209508 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.209471 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:20.505928 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.505855 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7"] Jul 24 18:14:20.506204 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.506188 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" containerName="istio-proxy" Jul 24 18:14:20.506249 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.506206 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" containerName="istio-proxy" Jul 24 18:14:20.506249 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.506219 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab402195-e4b9-4e6b-91e6-b1fcea40cdf7" containerName="revoke-keys" Jul 24 18:14:20.506249 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.506225 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab402195-e4b9-4e6b-91e6-b1fcea40cdf7" containerName="revoke-keys" Jul 24 18:14:20.506342 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.506276 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" containerName="istio-proxy" Jul 24 18:14:20.506342 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.506284 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab402195-e4b9-4e6b-91e6-b1fcea40cdf7" containerName="revoke-keys" Jul 24 18:14:20.509361 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.509340 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.511677 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.511656 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ad8b31a2-openshift-default-dockercfg-lddrt\"" Jul 24 18:14:20.526965 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.526920 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7"] Jul 24 18:14:20.542146 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.542113 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:20.552748 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.552717 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-token\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.552899 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.552765 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-workload-socket\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.552899 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.552794 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-envoy\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.552899 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.552859 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istiod-ca-cert\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.553101 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.552954 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbz65\" (UniqueName: \"kubernetes.io/projected/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-kube-api-access-sbz65\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.553101 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.553011 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-credential-socket\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.553101 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.553070 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-workload-certs\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.553256 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.553125 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-podinfo\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.553256 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.553193 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-data\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.653972 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.653915 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-token\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.654180 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.653981 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-workload-socket\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.654180 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.654010 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-envoy\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.654180 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.654039 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istiod-ca-cert\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.654389 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.654211 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbz65\" (UniqueName: \"kubernetes.io/projected/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-kube-api-access-sbz65\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.654389 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.654281 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-credential-socket\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.654389 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.654321 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-workload-certs\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.654389 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.654359 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-podinfo\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.654613 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.654415 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-workload-socket\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.654613 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.654428 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-data\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.654613 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.654567 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-credential-socket\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.654785 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.654766 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-workload-certs\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.654842 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.654786 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-data\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.654842 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.654814 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istiod-ca-cert\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.656763 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.656741 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-podinfo\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.656871 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.656747 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-envoy\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.662533 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.662503 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-token\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.662669 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.662647 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbz65\" (UniqueName: \"kubernetes.io/projected/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-kube-api-access-sbz65\") pod \"e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.814575 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.814496 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:20.819514 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.819493 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:20.956990 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.956961 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7"] Jul 24 18:14:20.958759 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:14:20.958731 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6f4fb4f_0d08_4ab2_a2f1_0c9c30576ea6.slice/crio-8459fe184d8c69f65273e8c0d455045cc4827022244ac54ee7e177910f4374aa WatchSource:0}: Error finding container 8459fe184d8c69f65273e8c0d455045cc4827022244ac54ee7e177910f4374aa: Status 404 returned error can't find the container with id 8459fe184d8c69f65273e8c0d455045cc4827022244ac54ee7e177910f4374aa Jul 24 18:14:20.960781 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.960748 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:14:20.960854 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.960818 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:14:20.960854 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:20.960845 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:14:21.090177 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:21.090081 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" event={"ID":"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6","Type":"ContainerStarted","Data":"7b487a6f250a16036386d9e71057931fca388901e6ca64938f4184f56f05c965"} Jul 24 18:14:21.090177 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:21.090139 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" event={"ID":"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6","Type":"ContainerStarted","Data":"8459fe184d8c69f65273e8c0d455045cc4827022244ac54ee7e177910f4374aa"} Jul 24 18:14:21.111976 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:21.111800 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" podStartSLOduration=1.111776503 podStartE2EDuration="1.111776503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:14:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:14:21.110592486 +0000 UTC m=+2056.723598352" watchObservedRunningTime="2026-07-24 18:14:21.111776503 +0000 UTC m=+2056.724782372" Jul 24 18:14:21.820350 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:21.820311 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:22.825508 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:22.825469 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:23.052234 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:23.052200 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:23.099224 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:23.099129 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:23.100188 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:23.100169 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:23.142769 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:23.142734 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:23.217523 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:23.217490 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:23.350265 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:23.350181 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:23.832393 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:23.832353 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:23.932917 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:23.932879 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:27.557827 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:27.557737 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:36.163719 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:36.163620 2561 generic.go:356] "Generic (PLEG): container finished" podID="e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" containerID="7b487a6f250a16036386d9e71057931fca388901e6ca64938f4184f56f05c965" exitCode=0 Jul 24 18:14:36.163719 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:36.163712 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" event={"ID":"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6","Type":"ContainerDied","Data":"7b487a6f250a16036386d9e71057931fca388901e6ca64938f4184f56f05c965"} Jul 24 18:14:36.164424 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:36.164206 2561 scope.go:117] "RemoveContainer" containerID="7b487a6f250a16036386d9e71057931fca388901e6ca64938f4184f56f05c965" Jul 24 18:14:36.165865 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:36.165830 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:14:36.165995 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:36.165906 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:14:36.165995 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:36.165964 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:14:36.651479 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:36.651427 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:14:37.173296 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:37.172287 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" event={"ID":"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6","Type":"ContainerStarted","Data":"29d852ac9d682edbc52b5a1e1cf564fbbf7ee81916841666b7c81a8e013f628d"} Jul 24 18:14:37.820266 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:37.820226 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:37.821440 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:37.821406 2561 patch_prober.go:28] interesting pod/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 24 18:14:37.821557 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:37.821461 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" podUID="e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 24 18:14:38.099910 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:38.099814 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:38.820303 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:38.820264 2561 patch_prober.go:28] interesting pod/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 24 18:14:38.820717 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:38.820327 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" podUID="e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 24 18:14:39.819985 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:39.819920 2561 patch_prober.go:28] interesting pod/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 24 18:14:39.820159 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:39.820007 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" podUID="e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 24 18:14:40.820448 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:40.820404 2561 patch_prober.go:28] interesting pod/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 24 18:14:40.820837 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:40.820465 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" podUID="e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 24 18:14:41.824175 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:41.824142 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:14:41.825081 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:14:41.825060 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:15:30.898272 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:15:30.898232 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:15:33.715400 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:15:33.715354 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:15:38.524904 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:15:38.524870 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:15:38.612337 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:15:38.612305 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:15:38.761146 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:15:38.761113 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:15:42.829691 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:15:42.829654 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:15:43.469634 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:15:43.469599 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:15:43.586544 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:15:43.586513 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:15:46.267998 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:15:46.267963 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:15:51.306490 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:15:51.306449 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:04.979660 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:04.979580 2561 scope.go:117] "RemoveContainer" containerID="ea209a1679664cb9f56060a874200008d9f07ec6a81448649cc5e715a055da15" Jul 24 18:16:04.987465 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:04.987441 2561 scope.go:117] "RemoveContainer" containerID="911f5e5acc219db6fce170e14cd90a35fc63bd41e9f6495c008f03430d0ca84c" Jul 24 18:16:09.001181 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:09.001147 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:10.038000 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:10.037965 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:16.056802 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:16.056754 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:22.021130 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:22.021093 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:22.093725 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:22.093689 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:22.308367 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:22.308277 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7"] Jul 24 18:16:22.308559 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:22.308534 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" podUID="e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" containerName="istio-proxy" containerID="cri-o://29d852ac9d682edbc52b5a1e1cf564fbbf7ee81916841666b7c81a8e013f628d" gracePeriod=30 Jul 24 18:16:22.364781 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:22.364749 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:22.593430 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:22.593340 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:23.723621 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:23.723588 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:23.819430 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:23.819401 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:23.950433 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:23.950398 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:24.781714 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:24.781681 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:24.879848 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:24.879819 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:26.482042 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:26.482012 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:26.826822 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:26.826743 2561 patch_prober.go:28] interesting pod/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 18:16:26.826822 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:26.826806 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" podUID="e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 18:16:27.558548 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.558523 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:16:27.635076 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.634999 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-credential-socket\") pod \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " Jul 24 18:16:27.635076 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.635031 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-token\") pod \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " Jul 24 18:16:27.635076 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.635049 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-envoy\") pod \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " Jul 24 18:16:27.635358 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.635081 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istiod-ca-cert\") pod \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " Jul 24 18:16:27.635358 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.635105 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-workload-certs\") pod \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " Jul 24 18:16:27.635358 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.635158 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-podinfo\") pod \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " Jul 24 18:16:27.635358 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.635210 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-workload-socket\") pod \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " Jul 24 18:16:27.635358 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.635238 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-data\") pod \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " Jul 24 18:16:27.635358 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.635264 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbz65\" (UniqueName: \"kubernetes.io/projected/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-kube-api-access-sbz65\") pod \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\" (UID: \"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6\") " Jul 24 18:16:27.635632 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.635380 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" (UID: "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:16:27.635632 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.635532 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" (UID: "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:16:27.635632 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.635543 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" (UID: "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:16:27.635795 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.635659 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:16:27.635795 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.635680 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:16:27.635795 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.635696 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:16:27.635795 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.635759 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" (UID: "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:16:27.636355 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.636333 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-data" (OuterVolumeSpecName: "istio-data") pod "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" (UID: "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:16:27.637393 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.637375 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" (UID: "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:16:27.637682 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.637660 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-token" (OuterVolumeSpecName: "istio-token") pod "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" (UID: "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:16:27.637749 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.637708 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-kube-api-access-sbz65" (OuterVolumeSpecName: "kube-api-access-sbz65") pod "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" (UID: "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6"). InnerVolumeSpecName "kube-api-access-sbz65". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:16:27.637898 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.637873 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" (UID: "e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:16:27.661185 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.661155 2561 generic.go:356] "Generic (PLEG): container finished" podID="e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" containerID="29d852ac9d682edbc52b5a1e1cf564fbbf7ee81916841666b7c81a8e013f628d" exitCode=0 Jul 24 18:16:27.661320 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.661204 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" event={"ID":"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6","Type":"ContainerDied","Data":"29d852ac9d682edbc52b5a1e1cf564fbbf7ee81916841666b7c81a8e013f628d"} Jul 24 18:16:27.661320 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.661215 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" Jul 24 18:16:27.661320 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.661243 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7" event={"ID":"e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6","Type":"ContainerDied","Data":"8459fe184d8c69f65273e8c0d455045cc4827022244ac54ee7e177910f4374aa"} Jul 24 18:16:27.661320 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.661264 2561 scope.go:117] "RemoveContainer" containerID="29d852ac9d682edbc52b5a1e1cf564fbbf7ee81916841666b7c81a8e013f628d" Jul 24 18:16:27.669526 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.669508 2561 scope.go:117] "RemoveContainer" containerID="7b487a6f250a16036386d9e71057931fca388901e6ca64938f4184f56f05c965" Jul 24 18:16:27.677217 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.677201 2561 scope.go:117] "RemoveContainer" containerID="29d852ac9d682edbc52b5a1e1cf564fbbf7ee81916841666b7c81a8e013f628d" Jul 24 18:16:27.677438 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:16:27.677419 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29d852ac9d682edbc52b5a1e1cf564fbbf7ee81916841666b7c81a8e013f628d\": container with ID starting with 29d852ac9d682edbc52b5a1e1cf564fbbf7ee81916841666b7c81a8e013f628d not found: ID does not exist" containerID="29d852ac9d682edbc52b5a1e1cf564fbbf7ee81916841666b7c81a8e013f628d" Jul 24 18:16:27.677509 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.677451 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29d852ac9d682edbc52b5a1e1cf564fbbf7ee81916841666b7c81a8e013f628d"} err="failed to get container status \"29d852ac9d682edbc52b5a1e1cf564fbbf7ee81916841666b7c81a8e013f628d\": rpc error: code = NotFound desc = could not find container \"29d852ac9d682edbc52b5a1e1cf564fbbf7ee81916841666b7c81a8e013f628d\": container with ID starting with 29d852ac9d682edbc52b5a1e1cf564fbbf7ee81916841666b7c81a8e013f628d not found: ID does not exist" Jul 24 18:16:27.677509 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.677475 2561 scope.go:117] "RemoveContainer" containerID="7b487a6f250a16036386d9e71057931fca388901e6ca64938f4184f56f05c965" Jul 24 18:16:27.677739 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:16:27.677717 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b487a6f250a16036386d9e71057931fca388901e6ca64938f4184f56f05c965\": container with ID starting with 7b487a6f250a16036386d9e71057931fca388901e6ca64938f4184f56f05c965 not found: ID does not exist" containerID="7b487a6f250a16036386d9e71057931fca388901e6ca64938f4184f56f05c965" Jul 24 18:16:27.677783 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.677746 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b487a6f250a16036386d9e71057931fca388901e6ca64938f4184f56f05c965"} err="failed to get container status \"7b487a6f250a16036386d9e71057931fca388901e6ca64938f4184f56f05c965\": rpc error: code = NotFound desc = could not find container \"7b487a6f250a16036386d9e71057931fca388901e6ca64938f4184f56f05c965\": container with ID starting with 7b487a6f250a16036386d9e71057931fca388901e6ca64938f4184f56f05c965 not found: ID does not exist" Jul 24 18:16:27.686735 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.686713 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7"] Jul 24 18:16:27.694969 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.694927 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ad8b31a2-openshift-default-7f4d499d4-5mtd7"] Jul 24 18:16:27.736847 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.736825 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:16:27.736847 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.736845 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:16:27.736993 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.736853 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:16:27.736993 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.736862 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:16:27.736993 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.736869 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:16:27.736993 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:27.736880 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbz65\" (UniqueName: \"kubernetes.io/projected/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6-kube-api-access-sbz65\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:16:28.940508 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:28.940474 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" path="/var/lib/kubelet/pods/e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6/volumes" Jul 24 18:16:32.562286 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:32.562250 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:32.638843 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:32.638804 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:33.293593 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:33.293563 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:39.923018 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:39.922962 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:40.039118 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:40.039088 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:40.139381 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:40.139340 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:50.688286 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:50.688253 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:51.062143 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:51.062064 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:16:59.016193 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:16:59.016146 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:05.009601 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:05.009567 2561 scope.go:117] "RemoveContainer" containerID="f9d23ba57102c00126b6c37d6594fb560da07e5cdef586558bdbc276d5d67a61" Jul 24 18:17:12.094910 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:12.094870 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:18.434484 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:18.434444 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:18.538634 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:18.538598 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:18.831124 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:18.831036 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:19.249130 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.249096 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv"] Jul 24 18:17:19.249519 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.249496 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" containerName="istio-proxy" Jul 24 18:17:19.249519 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.249519 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" containerName="istio-proxy" Jul 24 18:17:19.249804 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.249560 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" containerName="istio-proxy" Jul 24 18:17:19.249804 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.249569 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" containerName="istio-proxy" Jul 24 18:17:19.249804 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.249582 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" containerName="istio-proxy" Jul 24 18:17:19.249804 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.249591 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" containerName="istio-proxy" Jul 24 18:17:19.249804 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.249665 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e7d9b3b-055e-4d3e-bd56-52a11ac843e0" containerName="istio-proxy" Jul 24 18:17:19.249804 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.249676 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" containerName="istio-proxy" Jul 24 18:17:19.252527 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.252507 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.255125 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.255104 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6e3964-openshift-default-dockercfg-4nfvx\"" Jul 24 18:17:19.265009 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.264986 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv"] Jul 24 18:17:19.265178 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.265021 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:19.368494 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.368462 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-token\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.368675 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.368506 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-podinfo\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.368675 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.368525 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/517e1e6a-1fcd-4266-b954-ea6792d6a031-istiod-ca-cert\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.368675 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.368612 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-credential-socket\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.368675 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.368648 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-data\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.368675 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.368667 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-workload-certs\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.368843 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.368685 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-envoy\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.368843 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.368711 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-workload-socket\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.368843 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.368751 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6nxm\" (UniqueName: \"kubernetes.io/projected/517e1e6a-1fcd-4266-b954-ea6792d6a031-kube-api-access-h6nxm\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.470009 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.469976 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-credential-socket\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.470009 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.470009 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-data\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.470546 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.470027 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-workload-certs\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.470546 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.470042 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-envoy\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.470546 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.470062 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-workload-socket\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.470546 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.470195 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6nxm\" (UniqueName: \"kubernetes.io/projected/517e1e6a-1fcd-4266-b954-ea6792d6a031-kube-api-access-h6nxm\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.470546 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.470307 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-token\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.470546 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.470373 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-podinfo\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.470546 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.470400 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/517e1e6a-1fcd-4266-b954-ea6792d6a031-istiod-ca-cert\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.470546 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.470455 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-data\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.470546 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.470467 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-credential-socket\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.470546 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.470529 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-workload-certs\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.471077 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.470792 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-workload-socket\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.471133 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.471120 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/517e1e6a-1fcd-4266-b954-ea6792d6a031-istiod-ca-cert\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.472648 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.472624 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-envoy\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.472871 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.472853 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-podinfo\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.481501 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.481472 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6nxm\" (UniqueName: \"kubernetes.io/projected/517e1e6a-1fcd-4266-b954-ea6792d6a031-kube-api-access-h6nxm\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.482977 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.482948 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-token\") pod \"e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.563738 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.563659 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:19.570227 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.570198 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:19.720263 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.720233 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv"] Jul 24 18:17:19.720912 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:17:19.720887 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod517e1e6a_1fcd_4266_b954_ea6792d6a031.slice/crio-75327f1c3bd557316e975c64d6bed342326bcb36ff979852c70a2a14a6e34d90 WatchSource:0}: Error finding container 75327f1c3bd557316e975c64d6bed342326bcb36ff979852c70a2a14a6e34d90: Status 404 returned error can't find the container with id 75327f1c3bd557316e975c64d6bed342326bcb36ff979852c70a2a14a6e34d90 Jul 24 18:17:19.723020 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.722991 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:17:19.723072 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.723053 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:17:19.723113 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.723083 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:17:19.871138 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.871046 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" event={"ID":"517e1e6a-1fcd-4266-b954-ea6792d6a031","Type":"ContainerStarted","Data":"4c359893cf9c76fb760837c16dc1ce1d37370a0105fc29e064494b050cd89cba"} Jul 24 18:17:19.871283 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.871144 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" event={"ID":"517e1e6a-1fcd-4266-b954-ea6792d6a031","Type":"ContainerStarted","Data":"75327f1c3bd557316e975c64d6bed342326bcb36ff979852c70a2a14a6e34d90"} Jul 24 18:17:19.901247 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:19.901183 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" podStartSLOduration=0.901164109 podStartE2EDuration="901.164109ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:17:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:17:19.899291416 +0000 UTC m=+2235.512297496" watchObservedRunningTime="2026-07-24 18:17:19.901164109 +0000 UTC m=+2235.514169978" Jul 24 18:17:20.564468 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:20.564432 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:20.569015 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:20.568992 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:20.875301 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:20.875218 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:20.876228 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:20.876202 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:21.892346 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:21.892287 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:21.991050 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:21.991003 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:22.094412 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:22.094376 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:22.397471 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:22.397438 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:22.674573 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:22.674492 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:22.775131 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:22.775091 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:26.452909 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:26.452874 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:26.886629 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:26.886548 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:27.153960 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.153844 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz"] Jul 24 18:17:27.154238 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.154224 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6f4fb4f-0d08-4ab2-a2f1-0c9c30576ea6" containerName="istio-proxy" Jul 24 18:17:27.157242 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.157222 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.160794 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.160775 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-0f1d5b-openshift-default-dockercfg-w6kt9\"" Jul 24 18:17:27.175009 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.174987 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz"] Jul 24 18:17:27.194408 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.194386 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:27.312700 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.312663 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:27.339888 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.339852 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4160e38-9aaf-440e-9238-2b45fcb08391-istiod-ca-cert\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.340066 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.339900 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-podinfo\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.340066 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.339981 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-workload-socket\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.340066 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.340037 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-credential-socket\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.340066 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.340056 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-data\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.340286 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.340108 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8khkb\" (UniqueName: \"kubernetes.io/projected/f4160e38-9aaf-440e-9238-2b45fcb08391-kube-api-access-8khkb\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.340286 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.340188 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-workload-certs\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.340286 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.340212 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-envoy\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.340286 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.340252 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-token\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.441154 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.441121 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-workload-certs\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.441154 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.441153 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-envoy\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.441378 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.441177 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-token\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.441378 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.441198 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4160e38-9aaf-440e-9238-2b45fcb08391-istiod-ca-cert\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.441378 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.441221 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-podinfo\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.441378 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.441252 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-workload-socket\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.441378 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.441282 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-credential-socket\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.441378 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.441305 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-data\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.441378 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.441347 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8khkb\" (UniqueName: \"kubernetes.io/projected/f4160e38-9aaf-440e-9238-2b45fcb08391-kube-api-access-8khkb\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.441746 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.441617 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-workload-certs\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.441746 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.441682 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-workload-socket\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.441908 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.441882 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-data\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.441908 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.441901 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-credential-socket\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.442156 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.442136 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4160e38-9aaf-440e-9238-2b45fcb08391-istiod-ca-cert\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.443574 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.443553 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-envoy\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.443831 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.443808 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-podinfo\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.456894 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.456873 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8khkb\" (UniqueName: \"kubernetes.io/projected/f4160e38-9aaf-440e-9238-2b45fcb08391-kube-api-access-8khkb\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.457995 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.457978 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-token\") pod \"e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.470013 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.469991 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:27.611499 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.611477 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz"] Jul 24 18:17:27.613810 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:17:27.613787 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4160e38_9aaf_440e_9238_2b45fcb08391.slice/crio-df79da2b752a9fb1883c28c6b55866b2bab947e73749a82d37691a17d9c977f0 WatchSource:0}: Error finding container df79da2b752a9fb1883c28c6b55866b2bab947e73749a82d37691a17d9c977f0: Status 404 returned error can't find the container with id df79da2b752a9fb1883c28c6b55866b2bab947e73749a82d37691a17d9c977f0 Jul 24 18:17:27.615909 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.615877 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:17:27.616017 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.615960 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:17:27.616017 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.615993 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:17:27.908154 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.908054 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" event={"ID":"f4160e38-9aaf-440e-9238-2b45fcb08391","Type":"ContainerStarted","Data":"ac0aecc86c044f7bbd76ede3183207faf5df18fd9af74dc7ec51903cdd89021f"} Jul 24 18:17:27.908154 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.908122 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" event={"ID":"f4160e38-9aaf-440e-9238-2b45fcb08391","Type":"ContainerStarted","Data":"df79da2b752a9fb1883c28c6b55866b2bab947e73749a82d37691a17d9c977f0"} Jul 24 18:17:27.938701 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:27.938651 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" podStartSLOduration=0.938635208 podStartE2EDuration="938.635208ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:17:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:17:27.936317979 +0000 UTC m=+2243.549323845" watchObservedRunningTime="2026-07-24 18:17:27.938635208 +0000 UTC m=+2243.551641073" Jul 24 18:17:28.470124 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:28.470089 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:28.475145 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:28.475121 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:28.915616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:28.915505 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:28.916597 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:28.916565 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:30.039145 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:30.039112 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:30.171425 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:30.171393 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:30.354132 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:30.354053 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:30.792795 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:30.792762 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:31.123228 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:31.123152 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:31.275774 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:31.275737 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:34.471301 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:34.471268 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:35.877284 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:35.877188 2561 patch_prober.go:28] interesting pod/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 24 18:17:35.877900 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:35.877864 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" podUID="517e1e6a-1fcd-4266-b954-ea6792d6a031" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 24 18:17:35.948598 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:35.948563 2561 generic.go:356] "Generic (PLEG): container finished" podID="517e1e6a-1fcd-4266-b954-ea6792d6a031" containerID="4c359893cf9c76fb760837c16dc1ce1d37370a0105fc29e064494b050cd89cba" exitCode=0 Jul 24 18:17:35.948763 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:35.948640 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" event={"ID":"517e1e6a-1fcd-4266-b954-ea6792d6a031","Type":"ContainerDied","Data":"4c359893cf9c76fb760837c16dc1ce1d37370a0105fc29e064494b050cd89cba"} Jul 24 18:17:35.949092 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:35.949078 2561 scope.go:117] "RemoveContainer" containerID="4c359893cf9c76fb760837c16dc1ce1d37370a0105fc29e064494b050cd89cba" Jul 24 18:17:35.950172 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:35.950146 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:17:35.950240 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:35.950211 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:17:35.950277 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:35.950241 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:17:36.953696 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:36.953658 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" event={"ID":"517e1e6a-1fcd-4266-b954-ea6792d6a031","Type":"ContainerStarted","Data":"c3468f689fc3f9f9fa40ee8d1d9267177ddfe8bc0e9bc0d1c477ba69de766fed"} Jul 24 18:17:37.564754 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:37.564709 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:37.565007 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:37.564981 2561 patch_prober.go:28] interesting pod/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 24 18:17:37.565144 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:37.565054 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" podUID="517e1e6a-1fcd-4266-b954-ea6792d6a031" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 24 18:17:38.564414 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:38.564373 2561 patch_prober.go:28] interesting pod/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 24 18:17:38.564812 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:38.564443 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" podUID="517e1e6a-1fcd-4266-b954-ea6792d6a031" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 24 18:17:39.564422 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:39.564384 2561 patch_prober.go:28] interesting pod/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 24 18:17:39.564756 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:39.564446 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" podUID="517e1e6a-1fcd-4266-b954-ea6792d6a031" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 24 18:17:40.565124 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:40.565081 2561 patch_prober.go:28] interesting pod/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 24 18:17:40.565517 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:40.565165 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" podUID="517e1e6a-1fcd-4266-b954-ea6792d6a031" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 24 18:17:41.568115 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:41.568085 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:41.568550 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:41.568361 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:41.569006 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:41.568987 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:17:42.979987 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:42.979926 2561 generic.go:356] "Generic (PLEG): container finished" podID="f4160e38-9aaf-440e-9238-2b45fcb08391" containerID="ac0aecc86c044f7bbd76ede3183207faf5df18fd9af74dc7ec51903cdd89021f" exitCode=0 Jul 24 18:17:42.980469 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:42.980005 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" event={"ID":"f4160e38-9aaf-440e-9238-2b45fcb08391","Type":"ContainerDied","Data":"ac0aecc86c044f7bbd76ede3183207faf5df18fd9af74dc7ec51903cdd89021f"} Jul 24 18:17:42.980549 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:42.980520 2561 scope.go:117] "RemoveContainer" containerID="ac0aecc86c044f7bbd76ede3183207faf5df18fd9af74dc7ec51903cdd89021f" Jul 24 18:17:42.981477 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:42.981452 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:17:42.981536 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:42.981513 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:17:42.981571 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:42.981552 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:17:43.915740 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:43.915700 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:43.987715 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:43.987684 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" event={"ID":"f4160e38-9aaf-440e-9238-2b45fcb08391","Type":"ContainerStarted","Data":"005ece864ebc3c30fe87f527f26190f13dce987d70ba42c40cea599983597ccd"} Jul 24 18:17:44.470857 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:44.470813 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:44.472129 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:44.472101 2561 patch_prober.go:28] interesting pod/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 24 18:17:44.472356 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:44.472328 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" podUID="f4160e38-9aaf-440e-9238-2b45fcb08391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 24 18:17:45.471003 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:45.470931 2561 patch_prober.go:28] interesting pod/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 24 18:17:45.471492 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:45.471035 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" podUID="f4160e38-9aaf-440e-9238-2b45fcb08391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 24 18:17:46.470656 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:46.470625 2561 patch_prober.go:28] interesting pod/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 24 18:17:46.470826 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:46.470692 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" podUID="f4160e38-9aaf-440e-9238-2b45fcb08391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 24 18:17:46.805443 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:46.805351 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:17:47.470726 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:47.470692 2561 patch_prober.go:28] interesting pod/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 24 18:17:47.470912 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:47.470750 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" podUID="f4160e38-9aaf-440e-9238-2b45fcb08391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 24 18:17:48.474305 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:48.474271 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:48.474712 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:48.474635 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:17:48.475355 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:17:48.475333 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:18:05.029286 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:18:05.029253 2561 scope.go:117] "RemoveContainer" containerID="6dd9a4269198682634a0d89440992199d1fad1f84537aed154a06aebee870373" Jul 24 18:18:19.553438 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:18:19.553400 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:18:59.444428 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:18:59.444354 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:00.366140 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:00.366108 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:11.063464 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:11.063431 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:11.167553 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:11.167519 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:11.263972 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:11.263920 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:15.576921 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:15.576876 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:15.989017 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:15.988983 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:21.942058 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:21.942025 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:22.048537 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:22.048498 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:22.175262 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:22.175229 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv"] Jul 24 18:19:22.175503 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:22.175479 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" podUID="517e1e6a-1fcd-4266-b954-ea6792d6a031" containerName="istio-proxy" containerID="cri-o://c3468f689fc3f9f9fa40ee8d1d9267177ddfe8bc0e9bc0d1c477ba69de766fed" gracePeriod=30 Jul 24 18:19:22.201952 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:22.201858 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:22.435058 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:22.435025 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:26.570052 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:26.570018 2561 patch_prober.go:28] interesting pod/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 18:19:26.570427 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:26.570083 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" podUID="517e1e6a-1fcd-4266-b954-ea6792d6a031" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 18:19:27.420574 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.420551 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:19:27.425676 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.425648 2561 generic.go:356] "Generic (PLEG): container finished" podID="517e1e6a-1fcd-4266-b954-ea6792d6a031" containerID="c3468f689fc3f9f9fa40ee8d1d9267177ddfe8bc0e9bc0d1c477ba69de766fed" exitCode=0 Jul 24 18:19:27.425781 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.425689 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" Jul 24 18:19:27.425781 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.425718 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" event={"ID":"517e1e6a-1fcd-4266-b954-ea6792d6a031","Type":"ContainerDied","Data":"c3468f689fc3f9f9fa40ee8d1d9267177ddfe8bc0e9bc0d1c477ba69de766fed"} Jul 24 18:19:27.425781 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.425768 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv" event={"ID":"517e1e6a-1fcd-4266-b954-ea6792d6a031","Type":"ContainerDied","Data":"75327f1c3bd557316e975c64d6bed342326bcb36ff979852c70a2a14a6e34d90"} Jul 24 18:19:27.425889 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.425793 2561 scope.go:117] "RemoveContainer" containerID="c3468f689fc3f9f9fa40ee8d1d9267177ddfe8bc0e9bc0d1c477ba69de766fed" Jul 24 18:19:27.433777 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.433763 2561 scope.go:117] "RemoveContainer" containerID="4c359893cf9c76fb760837c16dc1ce1d37370a0105fc29e064494b050cd89cba" Jul 24 18:19:27.443087 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.443069 2561 scope.go:117] "RemoveContainer" containerID="c3468f689fc3f9f9fa40ee8d1d9267177ddfe8bc0e9bc0d1c477ba69de766fed" Jul 24 18:19:27.443426 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:19:27.443397 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3468f689fc3f9f9fa40ee8d1d9267177ddfe8bc0e9bc0d1c477ba69de766fed\": container with ID starting with c3468f689fc3f9f9fa40ee8d1d9267177ddfe8bc0e9bc0d1c477ba69de766fed not found: ID does not exist" containerID="c3468f689fc3f9f9fa40ee8d1d9267177ddfe8bc0e9bc0d1c477ba69de766fed" Jul 24 18:19:27.443510 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.443435 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3468f689fc3f9f9fa40ee8d1d9267177ddfe8bc0e9bc0d1c477ba69de766fed"} err="failed to get container status \"c3468f689fc3f9f9fa40ee8d1d9267177ddfe8bc0e9bc0d1c477ba69de766fed\": rpc error: code = NotFound desc = could not find container \"c3468f689fc3f9f9fa40ee8d1d9267177ddfe8bc0e9bc0d1c477ba69de766fed\": container with ID starting with c3468f689fc3f9f9fa40ee8d1d9267177ddfe8bc0e9bc0d1c477ba69de766fed not found: ID does not exist" Jul 24 18:19:27.443510 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.443468 2561 scope.go:117] "RemoveContainer" containerID="4c359893cf9c76fb760837c16dc1ce1d37370a0105fc29e064494b050cd89cba" Jul 24 18:19:27.444139 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:19:27.443844 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c359893cf9c76fb760837c16dc1ce1d37370a0105fc29e064494b050cd89cba\": container with ID starting with 4c359893cf9c76fb760837c16dc1ce1d37370a0105fc29e064494b050cd89cba not found: ID does not exist" containerID="4c359893cf9c76fb760837c16dc1ce1d37370a0105fc29e064494b050cd89cba" Jul 24 18:19:27.444246 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.444146 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c359893cf9c76fb760837c16dc1ce1d37370a0105fc29e064494b050cd89cba"} err="failed to get container status \"4c359893cf9c76fb760837c16dc1ce1d37370a0105fc29e064494b050cd89cba\": rpc error: code = NotFound desc = could not find container \"4c359893cf9c76fb760837c16dc1ce1d37370a0105fc29e064494b050cd89cba\": container with ID starting with 4c359893cf9c76fb760837c16dc1ce1d37370a0105fc29e064494b050cd89cba not found: ID does not exist" Jul 24 18:19:27.510409 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.510384 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-token\") pod \"517e1e6a-1fcd-4266-b954-ea6792d6a031\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " Jul 24 18:19:27.510628 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.510422 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-data\") pod \"517e1e6a-1fcd-4266-b954-ea6792d6a031\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " Jul 24 18:19:27.510628 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.510464 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-envoy\") pod \"517e1e6a-1fcd-4266-b954-ea6792d6a031\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " Jul 24 18:19:27.510628 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.510488 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/517e1e6a-1fcd-4266-b954-ea6792d6a031-istiod-ca-cert\") pod \"517e1e6a-1fcd-4266-b954-ea6792d6a031\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " Jul 24 18:19:27.510628 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.510535 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-workload-socket\") pod \"517e1e6a-1fcd-4266-b954-ea6792d6a031\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " Jul 24 18:19:27.510628 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.510555 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-podinfo\") pod \"517e1e6a-1fcd-4266-b954-ea6792d6a031\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " Jul 24 18:19:27.510628 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.510586 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-workload-certs\") pod \"517e1e6a-1fcd-4266-b954-ea6792d6a031\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " Jul 24 18:19:27.510980 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.510634 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-credential-socket\") pod \"517e1e6a-1fcd-4266-b954-ea6792d6a031\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " Jul 24 18:19:27.510980 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.510659 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6nxm\" (UniqueName: \"kubernetes.io/projected/517e1e6a-1fcd-4266-b954-ea6792d6a031-kube-api-access-h6nxm\") pod \"517e1e6a-1fcd-4266-b954-ea6792d6a031\" (UID: \"517e1e6a-1fcd-4266-b954-ea6792d6a031\") " Jul 24 18:19:27.511087 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.511010 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "517e1e6a-1fcd-4266-b954-ea6792d6a031" (UID: "517e1e6a-1fcd-4266-b954-ea6792d6a031"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:19:27.511387 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.511223 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "517e1e6a-1fcd-4266-b954-ea6792d6a031" (UID: "517e1e6a-1fcd-4266-b954-ea6792d6a031"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:19:27.511387 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.511241 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/517e1e6a-1fcd-4266-b954-ea6792d6a031-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "517e1e6a-1fcd-4266-b954-ea6792d6a031" (UID: "517e1e6a-1fcd-4266-b954-ea6792d6a031"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:19:27.511387 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.511353 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "517e1e6a-1fcd-4266-b954-ea6792d6a031" (UID: "517e1e6a-1fcd-4266-b954-ea6792d6a031"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:19:27.511560 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.511472 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-data" (OuterVolumeSpecName: "istio-data") pod "517e1e6a-1fcd-4266-b954-ea6792d6a031" (UID: "517e1e6a-1fcd-4266-b954-ea6792d6a031"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:19:27.513473 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.513433 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-token" (OuterVolumeSpecName: "istio-token") pod "517e1e6a-1fcd-4266-b954-ea6792d6a031" (UID: "517e1e6a-1fcd-4266-b954-ea6792d6a031"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:19:27.513473 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.513433 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/517e1e6a-1fcd-4266-b954-ea6792d6a031-kube-api-access-h6nxm" (OuterVolumeSpecName: "kube-api-access-h6nxm") pod "517e1e6a-1fcd-4266-b954-ea6792d6a031" (UID: "517e1e6a-1fcd-4266-b954-ea6792d6a031"). InnerVolumeSpecName "kube-api-access-h6nxm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:19:27.513473 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.513438 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "517e1e6a-1fcd-4266-b954-ea6792d6a031" (UID: "517e1e6a-1fcd-4266-b954-ea6792d6a031"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:19:27.513628 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.513539 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "517e1e6a-1fcd-4266-b954-ea6792d6a031" (UID: "517e1e6a-1fcd-4266-b954-ea6792d6a031"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:19:27.611704 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.611667 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:27.611704 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.611700 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:27.611704 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.611709 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:27.611704 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.611717 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/517e1e6a-1fcd-4266-b954-ea6792d6a031-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:27.612135 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.611725 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:27.612135 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.611733 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/517e1e6a-1fcd-4266-b954-ea6792d6a031-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:27.612135 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.611741 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:27.612135 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.611750 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/517e1e6a-1fcd-4266-b954-ea6792d6a031-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:27.612135 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.611758 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6nxm\" (UniqueName: \"kubernetes.io/projected/517e1e6a-1fcd-4266-b954-ea6792d6a031-kube-api-access-h6nxm\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:27.751179 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.751149 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv"] Jul 24 18:19:27.755734 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:27.755677 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6e3964-openshift-default-87b9b4bb4-48zxv"] Jul 24 18:19:28.940730 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:28.940698 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="517e1e6a-1fcd-4266-b954-ea6792d6a031" path="/var/lib/kubelet/pods/517e1e6a-1fcd-4266-b954-ea6792d6a031/volumes" Jul 24 18:19:33.148853 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:33.148819 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:33.250746 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:33.250712 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:33.354669 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:33.354635 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:37.705439 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:37.705406 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:38.116397 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:38.116321 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:44.694270 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:44.694235 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:44.803457 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:44.799070 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:44.969596 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:44.964531 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz"] Jul 24 18:19:44.969596 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:44.964785 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" podUID="f4160e38-9aaf-440e-9238-2b45fcb08391" containerName="istio-proxy" containerID="cri-o://005ece864ebc3c30fe87f527f26190f13dce987d70ba42c40cea599983597ccd" gracePeriod=30 Jul 24 18:19:44.990260 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:44.990232 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:45.241705 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:45.241611 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:45.579810 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:45.579731 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:45.888992 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:45.888899 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4"] Jul 24 18:19:45.889332 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:45.889200 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="517e1e6a-1fcd-4266-b954-ea6792d6a031" containerName="istio-proxy" Jul 24 18:19:45.889332 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:45.889213 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="517e1e6a-1fcd-4266-b954-ea6792d6a031" containerName="istio-proxy" Jul 24 18:19:45.889332 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:45.889273 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="517e1e6a-1fcd-4266-b954-ea6792d6a031" containerName="istio-proxy" Jul 24 18:19:45.889332 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:45.889281 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="517e1e6a-1fcd-4266-b954-ea6792d6a031" containerName="istio-proxy" Jul 24 18:19:45.892136 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:45.892118 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:45.895412 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:45.895389 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-013528-openshift-default-dockercfg-ccpps\"" Jul 24 18:19:45.909154 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:45.909134 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4"] Jul 24 18:19:45.924503 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:45.924482 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:46.071816 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.071781 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-credential-socket\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.071816 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.071820 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-workload-socket\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.072036 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.071841 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-podinfo\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.072036 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.071973 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-workload-certs\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.072036 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.072003 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjst7\" (UniqueName: \"kubernetes.io/projected/09af5645-5e0d-462a-a367-7bb84f1c546f-kube-api-access-kjst7\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.072139 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.072038 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09af5645-5e0d-462a-a367-7bb84f1c546f-istiod-ca-cert\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.072139 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.072066 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-data\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.072139 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.072084 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-token\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.072139 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.072126 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-envoy\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.082320 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.082292 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:46.172658 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.172585 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-credential-socket\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.172658 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.172621 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-workload-socket\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.172658 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.172641 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-podinfo\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.173005 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.172673 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-workload-certs\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.173005 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.172699 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjst7\" (UniqueName: \"kubernetes.io/projected/09af5645-5e0d-462a-a367-7bb84f1c546f-kube-api-access-kjst7\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.173005 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.172731 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09af5645-5e0d-462a-a367-7bb84f1c546f-istiod-ca-cert\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.173005 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.172750 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-data\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.173005 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.172775 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-token\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.173005 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.172802 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-envoy\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.173313 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.173062 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-credential-socket\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.173313 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.173129 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-workload-socket\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.173425 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.173326 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-data\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.173425 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.173393 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-workload-certs\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.173649 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.173628 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09af5645-5e0d-462a-a367-7bb84f1c546f-istiod-ca-cert\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.175068 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.175045 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-podinfo\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.175219 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.175204 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-envoy\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.181916 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.181892 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-token\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.182346 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.182330 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjst7\" (UniqueName: \"kubernetes.io/projected/09af5645-5e0d-462a-a367-7bb84f1c546f-kube-api-access-kjst7\") pod \"e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.202244 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.202222 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:46.341687 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.341657 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4"] Jul 24 18:19:46.344801 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:19:46.344774 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09af5645_5e0d_462a_a367_7bb84f1c546f.slice/crio-675ea7cbda287a8d8a9a827ed2d1cbc3ed099eb2829b58b550513a8be153d59d WatchSource:0}: Error finding container 675ea7cbda287a8d8a9a827ed2d1cbc3ed099eb2829b58b550513a8be153d59d: Status 404 returned error can't find the container with id 675ea7cbda287a8d8a9a827ed2d1cbc3ed099eb2829b58b550513a8be153d59d Jul 24 18:19:46.346817 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.346787 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:19:46.346908 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.346844 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:19:46.346908 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.346876 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:19:46.506914 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.506872 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" event={"ID":"09af5645-5e0d-462a-a367-7bb84f1c546f","Type":"ContainerStarted","Data":"403b7726f162f5ca479a0eb4f3621a1668cc37dd9ace29c5a2c3c460b078b31e"} Jul 24 18:19:46.507134 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.506924 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" event={"ID":"09af5645-5e0d-462a-a367-7bb84f1c546f","Type":"ContainerStarted","Data":"675ea7cbda287a8d8a9a827ed2d1cbc3ed099eb2829b58b550513a8be153d59d"} Jul 24 18:19:46.534546 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:46.534487 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" podStartSLOduration=1.534473009 podStartE2EDuration="1.534473009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:19:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:19:46.532501368 +0000 UTC m=+2382.145507235" watchObservedRunningTime="2026-07-24 18:19:46.534473009 +0000 UTC m=+2382.147478874" Jul 24 18:19:47.202972 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:47.202922 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:47.207589 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:47.207566 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:47.511364 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:47.511268 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:47.512288 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:47.512262 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:19:48.476582 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:48.476551 2561 patch_prober.go:28] interesting pod/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 18:19:48.476985 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:48.476601 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" podUID="f4160e38-9aaf-440e-9238-2b45fcb08391" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 18:19:48.511669 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:48.511640 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:48.612321 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:48.612287 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:48.712050 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:48.712019 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:48.809215 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:48.809100 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:49.337552 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:49.337511 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:49.456306 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:49.456276 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:50.214876 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.214853 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:19:50.309693 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.309621 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-workload-socket\") pod \"f4160e38-9aaf-440e-9238-2b45fcb08391\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " Jul 24 18:19:50.309693 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.309659 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-workload-certs\") pod \"f4160e38-9aaf-440e-9238-2b45fcb08391\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " Jul 24 18:19:50.309926 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.309699 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8khkb\" (UniqueName: \"kubernetes.io/projected/f4160e38-9aaf-440e-9238-2b45fcb08391-kube-api-access-8khkb\") pod \"f4160e38-9aaf-440e-9238-2b45fcb08391\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " Jul 24 18:19:50.309926 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.309716 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4160e38-9aaf-440e-9238-2b45fcb08391-istiod-ca-cert\") pod \"f4160e38-9aaf-440e-9238-2b45fcb08391\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " Jul 24 18:19:50.309926 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.309738 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-token\") pod \"f4160e38-9aaf-440e-9238-2b45fcb08391\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " Jul 24 18:19:50.309926 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.309773 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-data\") pod \"f4160e38-9aaf-440e-9238-2b45fcb08391\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " Jul 24 18:19:50.309926 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.309794 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-envoy\") pod \"f4160e38-9aaf-440e-9238-2b45fcb08391\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " Jul 24 18:19:50.309926 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.309817 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-podinfo\") pod \"f4160e38-9aaf-440e-9238-2b45fcb08391\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " Jul 24 18:19:50.309926 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.309860 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-credential-socket\") pod \"f4160e38-9aaf-440e-9238-2b45fcb08391\" (UID: \"f4160e38-9aaf-440e-9238-2b45fcb08391\") " Jul 24 18:19:50.309926 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.309914 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f4160e38-9aaf-440e-9238-2b45fcb08391" (UID: "f4160e38-9aaf-440e-9238-2b45fcb08391"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:19:50.310351 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.310134 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f4160e38-9aaf-440e-9238-2b45fcb08391" (UID: "f4160e38-9aaf-440e-9238-2b45fcb08391"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:19:50.310351 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.310177 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:50.310351 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.310208 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4160e38-9aaf-440e-9238-2b45fcb08391-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f4160e38-9aaf-440e-9238-2b45fcb08391" (UID: "f4160e38-9aaf-440e-9238-2b45fcb08391"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:19:50.310505 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.310472 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f4160e38-9aaf-440e-9238-2b45fcb08391" (UID: "f4160e38-9aaf-440e-9238-2b45fcb08391"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:19:50.310759 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.310732 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-data" (OuterVolumeSpecName: "istio-data") pod "f4160e38-9aaf-440e-9238-2b45fcb08391" (UID: "f4160e38-9aaf-440e-9238-2b45fcb08391"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:19:50.312066 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.312044 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f4160e38-9aaf-440e-9238-2b45fcb08391" (UID: "f4160e38-9aaf-440e-9238-2b45fcb08391"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:19:50.312286 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.312070 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-token" (OuterVolumeSpecName: "istio-token") pod "f4160e38-9aaf-440e-9238-2b45fcb08391" (UID: "f4160e38-9aaf-440e-9238-2b45fcb08391"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:19:50.312286 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.312096 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f4160e38-9aaf-440e-9238-2b45fcb08391" (UID: "f4160e38-9aaf-440e-9238-2b45fcb08391"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:19:50.312286 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.312109 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4160e38-9aaf-440e-9238-2b45fcb08391-kube-api-access-8khkb" (OuterVolumeSpecName: "kube-api-access-8khkb") pod "f4160e38-9aaf-440e-9238-2b45fcb08391" (UID: "f4160e38-9aaf-440e-9238-2b45fcb08391"). InnerVolumeSpecName "kube-api-access-8khkb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:19:50.411443 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.411411 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:50.411443 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.411436 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:50.411443 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.411445 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:50.411670 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.411454 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:50.411670 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.411464 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4160e38-9aaf-440e-9238-2b45fcb08391-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:50.411670 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.411472 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8khkb\" (UniqueName: \"kubernetes.io/projected/f4160e38-9aaf-440e-9238-2b45fcb08391-kube-api-access-8khkb\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:50.411670 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.411481 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4160e38-9aaf-440e-9238-2b45fcb08391-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:50.411670 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.411489 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4160e38-9aaf-440e-9238-2b45fcb08391-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:19:50.524494 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.524464 2561 generic.go:356] "Generic (PLEG): container finished" podID="f4160e38-9aaf-440e-9238-2b45fcb08391" containerID="005ece864ebc3c30fe87f527f26190f13dce987d70ba42c40cea599983597ccd" exitCode=0 Jul 24 18:19:50.524644 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.524511 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" Jul 24 18:19:50.524644 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.524548 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" event={"ID":"f4160e38-9aaf-440e-9238-2b45fcb08391","Type":"ContainerDied","Data":"005ece864ebc3c30fe87f527f26190f13dce987d70ba42c40cea599983597ccd"} Jul 24 18:19:50.524644 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.524602 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz" event={"ID":"f4160e38-9aaf-440e-9238-2b45fcb08391","Type":"ContainerDied","Data":"df79da2b752a9fb1883c28c6b55866b2bab947e73749a82d37691a17d9c977f0"} Jul 24 18:19:50.524644 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.524625 2561 scope.go:117] "RemoveContainer" containerID="005ece864ebc3c30fe87f527f26190f13dce987d70ba42c40cea599983597ccd" Jul 24 18:19:50.533172 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.533156 2561 scope.go:117] "RemoveContainer" containerID="ac0aecc86c044f7bbd76ede3183207faf5df18fd9af74dc7ec51903cdd89021f" Jul 24 18:19:50.542372 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.542352 2561 scope.go:117] "RemoveContainer" containerID="005ece864ebc3c30fe87f527f26190f13dce987d70ba42c40cea599983597ccd" Jul 24 18:19:50.542780 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:19:50.542762 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"005ece864ebc3c30fe87f527f26190f13dce987d70ba42c40cea599983597ccd\": container with ID starting with 005ece864ebc3c30fe87f527f26190f13dce987d70ba42c40cea599983597ccd not found: ID does not exist" containerID="005ece864ebc3c30fe87f527f26190f13dce987d70ba42c40cea599983597ccd" Jul 24 18:19:50.542853 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.542790 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"005ece864ebc3c30fe87f527f26190f13dce987d70ba42c40cea599983597ccd"} err="failed to get container status \"005ece864ebc3c30fe87f527f26190f13dce987d70ba42c40cea599983597ccd\": rpc error: code = NotFound desc = could not find container \"005ece864ebc3c30fe87f527f26190f13dce987d70ba42c40cea599983597ccd\": container with ID starting with 005ece864ebc3c30fe87f527f26190f13dce987d70ba42c40cea599983597ccd not found: ID does not exist" Jul 24 18:19:50.542853 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.542807 2561 scope.go:117] "RemoveContainer" containerID="ac0aecc86c044f7bbd76ede3183207faf5df18fd9af74dc7ec51903cdd89021f" Jul 24 18:19:50.543094 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:19:50.543077 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac0aecc86c044f7bbd76ede3183207faf5df18fd9af74dc7ec51903cdd89021f\": container with ID starting with ac0aecc86c044f7bbd76ede3183207faf5df18fd9af74dc7ec51903cdd89021f not found: ID does not exist" containerID="ac0aecc86c044f7bbd76ede3183207faf5df18fd9af74dc7ec51903cdd89021f" Jul 24 18:19:50.543141 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.543099 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac0aecc86c044f7bbd76ede3183207faf5df18fd9af74dc7ec51903cdd89021f"} err="failed to get container status \"ac0aecc86c044f7bbd76ede3183207faf5df18fd9af74dc7ec51903cdd89021f\": rpc error: code = NotFound desc = could not find container \"ac0aecc86c044f7bbd76ede3183207faf5df18fd9af74dc7ec51903cdd89021f\": container with ID starting with ac0aecc86c044f7bbd76ede3183207faf5df18fd9af74dc7ec51903cdd89021f not found: ID does not exist" Jul 24 18:19:50.545862 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.545838 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz"] Jul 24 18:19:50.552318 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.552297 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-0f1d5b-openshift-default-5fff9bb86d-v7pnz"] Jul 24 18:19:50.939804 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:50.939771 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4160e38-9aaf-440e-9238-2b45fcb08391" path="/var/lib/kubelet/pods/f4160e38-9aaf-440e-9238-2b45fcb08391/volumes" Jul 24 18:19:53.019216 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.019189 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:53.495988 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.495908 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:53.795099 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.795020 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9"] Jul 24 18:19:53.795337 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.795325 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="517e1e6a-1fcd-4266-b954-ea6792d6a031" containerName="istio-proxy" Jul 24 18:19:53.795384 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.795339 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="517e1e6a-1fcd-4266-b954-ea6792d6a031" containerName="istio-proxy" Jul 24 18:19:53.795384 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.795353 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4160e38-9aaf-440e-9238-2b45fcb08391" containerName="istio-proxy" Jul 24 18:19:53.795384 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.795359 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4160e38-9aaf-440e-9238-2b45fcb08391" containerName="istio-proxy" Jul 24 18:19:53.795384 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.795365 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4160e38-9aaf-440e-9238-2b45fcb08391" containerName="istio-proxy" Jul 24 18:19:53.795384 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.795370 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4160e38-9aaf-440e-9238-2b45fcb08391" containerName="istio-proxy" Jul 24 18:19:53.795534 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.795427 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4160e38-9aaf-440e-9238-2b45fcb08391" containerName="istio-proxy" Jul 24 18:19:53.795534 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.795436 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4160e38-9aaf-440e-9238-2b45fcb08391" containerName="istio-proxy" Jul 24 18:19:53.799573 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.799546 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.802372 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.802348 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-d1a9df-openshift-default-dockercfg-272wp\"" Jul 24 18:19:53.812929 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.812908 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9"] Jul 24 18:19:53.841095 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.841058 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-data\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.841343 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.841319 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-credential-socket\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.841498 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.841482 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-envoy\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.841650 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.841628 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p44pk\" (UniqueName: \"kubernetes.io/projected/ecfb0396-718e-40a0-a4fd-271298f58b15-kube-api-access-p44pk\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.841783 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.841689 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-workload-socket\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.841783 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.841720 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-workload-certs\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.841783 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.841776 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-podinfo\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.841969 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.841812 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-token\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.841969 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.841843 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecfb0396-718e-40a0-a4fd-271298f58b15-istiod-ca-cert\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.843998 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.843974 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:53.942328 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.942292 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecfb0396-718e-40a0-a4fd-271298f58b15-istiod-ca-cert\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.942534 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.942351 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-data\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.942534 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.942393 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-credential-socket\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.942534 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.942417 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-envoy\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.942534 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.942458 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p44pk\" (UniqueName: \"kubernetes.io/projected/ecfb0396-718e-40a0-a4fd-271298f58b15-kube-api-access-p44pk\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.942534 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.942513 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-workload-socket\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.942892 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.942544 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-workload-certs\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.942892 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.942603 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-podinfo\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.942892 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.942638 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-token\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.943087 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.942960 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-workload-certs\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.943192 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.943140 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecfb0396-718e-40a0-a4fd-271298f58b15-istiod-ca-cert\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.943326 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.943164 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-workload-socket\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.943326 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.943272 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-data\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.943481 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.943343 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:53.943536 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.943490 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-credential-socket\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.945266 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.945248 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-podinfo\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.945378 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.945279 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-envoy\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.950341 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.950319 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p44pk\" (UniqueName: \"kubernetes.io/projected/ecfb0396-718e-40a0-a4fd-271298f58b15-kube-api-access-p44pk\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:53.950654 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:53.950633 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-token\") pod \"e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:54.110573 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:54.110500 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:54.239535 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:54.239442 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9"] Jul 24 18:19:54.242196 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:19:54.242167 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecfb0396_718e_40a0_a4fd_271298f58b15.slice/crio-651bb4aa3f7341808e551bac024bbb7ed36092c3496c10c0b9105e0a93d9dca4 WatchSource:0}: Error finding container 651bb4aa3f7341808e551bac024bbb7ed36092c3496c10c0b9105e0a93d9dca4: Status 404 returned error can't find the container with id 651bb4aa3f7341808e551bac024bbb7ed36092c3496c10c0b9105e0a93d9dca4 Jul 24 18:19:54.244387 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:54.244353 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:19:54.244501 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:54.244414 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:19:54.244501 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:54.244441 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:19:54.543000 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:54.542966 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" event={"ID":"ecfb0396-718e-40a0-a4fd-271298f58b15","Type":"ContainerStarted","Data":"ff7a1d3c8e44705758573519bcafca2050eb32f53e0b7c2020ca17b1cf28b0f8"} Jul 24 18:19:54.543164 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:54.543017 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" event={"ID":"ecfb0396-718e-40a0-a4fd-271298f58b15","Type":"ContainerStarted","Data":"651bb4aa3f7341808e551bac024bbb7ed36092c3496c10c0b9105e0a93d9dca4"} Jul 24 18:19:54.566429 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:54.566378 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" podStartSLOduration=1.5663618289999999 podStartE2EDuration="1.566361829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:19:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:19:54.565207406 +0000 UTC m=+2390.178213306" watchObservedRunningTime="2026-07-24 18:19:54.566361829 +0000 UTC m=+2390.179367695" Jul 24 18:19:55.111528 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:55.111490 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:55.116562 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:55.116532 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:55.546766 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:55.546738 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:55.547736 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:55.547720 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:19:56.618495 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:56.618462 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:56.719427 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:56.719393 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:56.811044 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:56.811012 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:57.282061 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:57.282008 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:57.426524 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:57.426491 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:19:57.517804 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:19:57.517774 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:01.049472 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:01.049379 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:03.370517 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:03.370067 2561 patch_prober.go:28] interesting pod/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": read tcp 10.133.0.2:48170->10.133.0.37:15021: read: connection reset by peer" start-of-body= Jul 24 18:20:03.370517 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:03.370128 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" podUID="09af5645-5e0d-462a-a367-7bb84f1c546f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": read tcp 10.133.0.2:48170->10.133.0.37:15021: read: connection reset by peer" Jul 24 18:20:03.582763 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:03.582661 2561 generic.go:356] "Generic (PLEG): container finished" podID="09af5645-5e0d-462a-a367-7bb84f1c546f" containerID="403b7726f162f5ca479a0eb4f3621a1668cc37dd9ace29c5a2c3c460b078b31e" exitCode=0 Jul 24 18:20:03.582967 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:03.582772 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" event={"ID":"09af5645-5e0d-462a-a367-7bb84f1c546f","Type":"ContainerDied","Data":"403b7726f162f5ca479a0eb4f3621a1668cc37dd9ace29c5a2c3c460b078b31e"} Jul 24 18:20:03.583292 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:03.583268 2561 scope.go:117] "RemoveContainer" containerID="403b7726f162f5ca479a0eb4f3621a1668cc37dd9ace29c5a2c3c460b078b31e" Jul 24 18:20:03.585003 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:03.584551 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:20:03.585003 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:03.584634 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:20:03.585003 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:03.584683 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:20:04.604546 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:04.604502 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" event={"ID":"09af5645-5e0d-462a-a367-7bb84f1c546f","Type":"ContainerStarted","Data":"2c6df6a1ca52e8b3152f2cc2687a36b8d7ecdadcee398f66aebcc54a6be3b8ad"} Jul 24 18:20:05.051532 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:05.051503 2561 scope.go:117] "RemoveContainer" containerID="681721aff2cf01f6b2d723b3c541b3abb0589340f4ea69270649e63dd8a90ea2" Jul 24 18:20:05.203454 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:05.202644 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:20:05.203454 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:05.203105 2561 patch_prober.go:28] interesting pod/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 24 18:20:05.203454 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:05.203170 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" podUID="09af5645-5e0d-462a-a367-7bb84f1c546f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 24 18:20:06.203689 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:06.203649 2561 patch_prober.go:28] interesting pod/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 24 18:20:06.204753 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:06.204275 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" podUID="09af5645-5e0d-462a-a367-7bb84f1c546f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 24 18:20:07.203474 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:07.203016 2561 patch_prober.go:28] interesting pod/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 24 18:20:07.203474 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:07.203082 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" podUID="09af5645-5e0d-462a-a367-7bb84f1c546f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 24 18:20:08.202751 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:08.202709 2561 patch_prober.go:28] interesting pod/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 24 18:20:08.203270 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:08.202771 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" podUID="09af5645-5e0d-462a-a367-7bb84f1c546f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 24 18:20:09.207109 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:09.207074 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:20:09.207529 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:09.207483 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:20:09.208180 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:09.208154 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:20:10.548188 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:10.548096 2561 patch_prober.go:28] interesting pod/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 24 18:20:10.548188 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:10.548157 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" podUID="ecfb0396-718e-40a0-a4fd-271298f58b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 24 18:20:10.637909 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:10.637873 2561 generic.go:356] "Generic (PLEG): container finished" podID="ecfb0396-718e-40a0-a4fd-271298f58b15" containerID="ff7a1d3c8e44705758573519bcafca2050eb32f53e0b7c2020ca17b1cf28b0f8" exitCode=0 Jul 24 18:20:10.638097 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:10.637969 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" event={"ID":"ecfb0396-718e-40a0-a4fd-271298f58b15","Type":"ContainerDied","Data":"ff7a1d3c8e44705758573519bcafca2050eb32f53e0b7c2020ca17b1cf28b0f8"} Jul 24 18:20:10.638387 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:10.638370 2561 scope.go:117] "RemoveContainer" containerID="ff7a1d3c8e44705758573519bcafca2050eb32f53e0b7c2020ca17b1cf28b0f8" Jul 24 18:20:10.639388 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:10.639360 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:20:10.639481 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:10.639444 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:20:10.639542 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:10.639489 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:20:11.644469 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:11.644438 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" event={"ID":"ecfb0396-718e-40a0-a4fd-271298f58b15","Type":"ContainerStarted","Data":"e5f3472d9d3931bcf6615488f127316ddf0bb06a6cfc59c165eea0864221d7d8"} Jul 24 18:20:12.111388 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:12.111343 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:20:12.111655 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:12.111630 2561 patch_prober.go:28] interesting pod/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 24 18:20:12.111709 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:12.111686 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" podUID="ecfb0396-718e-40a0-a4fd-271298f58b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 24 18:20:13.111346 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:13.111311 2561 patch_prober.go:28] interesting pod/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 24 18:20:13.111731 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:13.111366 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" podUID="ecfb0396-718e-40a0-a4fd-271298f58b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 24 18:20:14.111313 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:14.111276 2561 patch_prober.go:28] interesting pod/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 24 18:20:14.111498 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:14.111333 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" podUID="ecfb0396-718e-40a0-a4fd-271298f58b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 24 18:20:15.111664 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:15.111629 2561 patch_prober.go:28] interesting pod/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 24 18:20:15.112062 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:15.111691 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" podUID="ecfb0396-718e-40a0-a4fd-271298f58b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 24 18:20:16.115499 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:16.115469 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:20:16.116015 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:16.115864 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:20:16.116570 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:16.116552 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:20:24.743370 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:24.743329 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:24.850392 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:24.850361 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:24.948052 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:24.948017 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:29.288037 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:29.287955 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:29.711619 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:29.711582 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:35.579198 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:35.579161 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:35.687323 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:35.687291 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:35.879058 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:35.878968 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4"] Jul 24 18:20:35.879209 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:35.879181 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" podUID="09af5645-5e0d-462a-a367-7bb84f1c546f" containerName="istio-proxy" containerID="cri-o://2c6df6a1ca52e8b3152f2cc2687a36b8d7ecdadcee398f66aebcc54a6be3b8ad" gracePeriod=30 Jul 24 18:20:35.904985 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:35.904954 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:36.014193 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:36.014159 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:36.111835 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:36.111802 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:39.210476 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:39.210431 2561 patch_prober.go:28] interesting pod/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 18:20:39.210850 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:39.210489 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" podUID="09af5645-5e0d-462a-a367-7bb84f1c546f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 18:20:41.127721 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.127696 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:20:41.176563 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.176474 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-workload-socket\") pod \"09af5645-5e0d-462a-a367-7bb84f1c546f\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " Jul 24 18:20:41.176563 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.176539 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-token\") pod \"09af5645-5e0d-462a-a367-7bb84f1c546f\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " Jul 24 18:20:41.176563 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.176563 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-envoy\") pod \"09af5645-5e0d-462a-a367-7bb84f1c546f\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " Jul 24 18:20:41.176866 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.176589 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-data\") pod \"09af5645-5e0d-462a-a367-7bb84f1c546f\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " Jul 24 18:20:41.176866 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.176633 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-podinfo\") pod \"09af5645-5e0d-462a-a367-7bb84f1c546f\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " Jul 24 18:20:41.176866 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.176666 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjst7\" (UniqueName: \"kubernetes.io/projected/09af5645-5e0d-462a-a367-7bb84f1c546f-kube-api-access-kjst7\") pod \"09af5645-5e0d-462a-a367-7bb84f1c546f\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " Jul 24 18:20:41.176866 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.176719 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-workload-certs\") pod \"09af5645-5e0d-462a-a367-7bb84f1c546f\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " Jul 24 18:20:41.176866 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.176755 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-credential-socket\") pod \"09af5645-5e0d-462a-a367-7bb84f1c546f\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " Jul 24 18:20:41.176866 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.176779 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09af5645-5e0d-462a-a367-7bb84f1c546f-istiod-ca-cert\") pod \"09af5645-5e0d-462a-a367-7bb84f1c546f\" (UID: \"09af5645-5e0d-462a-a367-7bb84f1c546f\") " Jul 24 18:20:41.176866 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.176796 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "09af5645-5e0d-462a-a367-7bb84f1c546f" (UID: "09af5645-5e0d-462a-a367-7bb84f1c546f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:20:41.177334 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.177036 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "09af5645-5e0d-462a-a367-7bb84f1c546f" (UID: "09af5645-5e0d-462a-a367-7bb84f1c546f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:20:41.177334 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.177076 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:20:41.177334 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.177114 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "09af5645-5e0d-462a-a367-7bb84f1c546f" (UID: "09af5645-5e0d-462a-a367-7bb84f1c546f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:20:41.177334 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.177315 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09af5645-5e0d-462a-a367-7bb84f1c546f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "09af5645-5e0d-462a-a367-7bb84f1c546f" (UID: "09af5645-5e0d-462a-a367-7bb84f1c546f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:20:41.177562 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.177522 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-data" (OuterVolumeSpecName: "istio-data") pod "09af5645-5e0d-462a-a367-7bb84f1c546f" (UID: "09af5645-5e0d-462a-a367-7bb84f1c546f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:20:41.178930 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.178893 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-token" (OuterVolumeSpecName: "istio-token") pod "09af5645-5e0d-462a-a367-7bb84f1c546f" (UID: "09af5645-5e0d-462a-a367-7bb84f1c546f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:20:41.179070 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.178967 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09af5645-5e0d-462a-a367-7bb84f1c546f-kube-api-access-kjst7" (OuterVolumeSpecName: "kube-api-access-kjst7") pod "09af5645-5e0d-462a-a367-7bb84f1c546f" (UID: "09af5645-5e0d-462a-a367-7bb84f1c546f"). InnerVolumeSpecName "kube-api-access-kjst7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:20:41.179322 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.179297 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "09af5645-5e0d-462a-a367-7bb84f1c546f" (UID: "09af5645-5e0d-462a-a367-7bb84f1c546f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:20:41.179369 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.179298 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "09af5645-5e0d-462a-a367-7bb84f1c546f" (UID: "09af5645-5e0d-462a-a367-7bb84f1c546f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:20:41.278303 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.278265 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:20:41.278303 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.278294 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:20:41.278303 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.278302 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:20:41.278303 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.278313 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09af5645-5e0d-462a-a367-7bb84f1c546f-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:20:41.278573 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.278322 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjst7\" (UniqueName: \"kubernetes.io/projected/09af5645-5e0d-462a-a367-7bb84f1c546f-kube-api-access-kjst7\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:20:41.278573 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.278331 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:20:41.278573 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.278340 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09af5645-5e0d-462a-a367-7bb84f1c546f-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:20:41.278573 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.278348 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09af5645-5e0d-462a-a367-7bb84f1c546f-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:20:41.766196 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.766160 2561 generic.go:356] "Generic (PLEG): container finished" podID="09af5645-5e0d-462a-a367-7bb84f1c546f" containerID="2c6df6a1ca52e8b3152f2cc2687a36b8d7ecdadcee398f66aebcc54a6be3b8ad" exitCode=0 Jul 24 18:20:41.766373 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.766213 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" Jul 24 18:20:41.766373 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.766251 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" event={"ID":"09af5645-5e0d-462a-a367-7bb84f1c546f","Type":"ContainerDied","Data":"2c6df6a1ca52e8b3152f2cc2687a36b8d7ecdadcee398f66aebcc54a6be3b8ad"} Jul 24 18:20:41.766373 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.766306 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4" event={"ID":"09af5645-5e0d-462a-a367-7bb84f1c546f","Type":"ContainerDied","Data":"675ea7cbda287a8d8a9a827ed2d1cbc3ed099eb2829b58b550513a8be153d59d"} Jul 24 18:20:41.766373 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.766329 2561 scope.go:117] "RemoveContainer" containerID="2c6df6a1ca52e8b3152f2cc2687a36b8d7ecdadcee398f66aebcc54a6be3b8ad" Jul 24 18:20:41.780118 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.780085 2561 scope.go:117] "RemoveContainer" containerID="403b7726f162f5ca479a0eb4f3621a1668cc37dd9ace29c5a2c3c460b078b31e" Jul 24 18:20:41.788170 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.788144 2561 scope.go:117] "RemoveContainer" containerID="2c6df6a1ca52e8b3152f2cc2687a36b8d7ecdadcee398f66aebcc54a6be3b8ad" Jul 24 18:20:41.788436 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:20:41.788415 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c6df6a1ca52e8b3152f2cc2687a36b8d7ecdadcee398f66aebcc54a6be3b8ad\": container with ID starting with 2c6df6a1ca52e8b3152f2cc2687a36b8d7ecdadcee398f66aebcc54a6be3b8ad not found: ID does not exist" containerID="2c6df6a1ca52e8b3152f2cc2687a36b8d7ecdadcee398f66aebcc54a6be3b8ad" Jul 24 18:20:41.788531 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.788446 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c6df6a1ca52e8b3152f2cc2687a36b8d7ecdadcee398f66aebcc54a6be3b8ad"} err="failed to get container status \"2c6df6a1ca52e8b3152f2cc2687a36b8d7ecdadcee398f66aebcc54a6be3b8ad\": rpc error: code = NotFound desc = could not find container \"2c6df6a1ca52e8b3152f2cc2687a36b8d7ecdadcee398f66aebcc54a6be3b8ad\": container with ID starting with 2c6df6a1ca52e8b3152f2cc2687a36b8d7ecdadcee398f66aebcc54a6be3b8ad not found: ID does not exist" Jul 24 18:20:41.788531 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.788464 2561 scope.go:117] "RemoveContainer" containerID="403b7726f162f5ca479a0eb4f3621a1668cc37dd9ace29c5a2c3c460b078b31e" Jul 24 18:20:41.788707 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:20:41.788692 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"403b7726f162f5ca479a0eb4f3621a1668cc37dd9ace29c5a2c3c460b078b31e\": container with ID starting with 403b7726f162f5ca479a0eb4f3621a1668cc37dd9ace29c5a2c3c460b078b31e not found: ID does not exist" containerID="403b7726f162f5ca479a0eb4f3621a1668cc37dd9ace29c5a2c3c460b078b31e" Jul 24 18:20:41.788751 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.788709 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"403b7726f162f5ca479a0eb4f3621a1668cc37dd9ace29c5a2c3c460b078b31e"} err="failed to get container status \"403b7726f162f5ca479a0eb4f3621a1668cc37dd9ace29c5a2c3c460b078b31e\": rpc error: code = NotFound desc = could not find container \"403b7726f162f5ca479a0eb4f3621a1668cc37dd9ace29c5a2c3c460b078b31e\": container with ID starting with 403b7726f162f5ca479a0eb4f3621a1668cc37dd9ace29c5a2c3c460b078b31e not found: ID does not exist" Jul 24 18:20:41.792443 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.792419 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4"] Jul 24 18:20:41.797430 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:41.797409 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-013528-openshift-default-5d5c4fff46-g47t4"] Jul 24 18:20:42.939885 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:42.939849 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09af5645-5e0d-462a-a367-7bb84f1c546f" path="/var/lib/kubelet/pods/09af5645-5e0d-462a-a367-7bb84f1c546f/volumes" Jul 24 18:20:51.724839 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:51.724805 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:51.828171 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:51.828137 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:51.928713 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:51.928665 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:56.309322 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:56.309277 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:20:56.711810 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:20:56.711780 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:02.619014 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:02.618978 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:02.719877 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:02.719846 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:02.827169 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:02.827138 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:03.013604 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.013564 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9"] Jul 24 18:21:03.013811 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.013787 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" podUID="ecfb0396-718e-40a0-a4fd-271298f58b15" containerName="istio-proxy" containerID="cri-o://e5f3472d9d3931bcf6615488f127316ddf0bb06a6cfc59c165eea0864221d7d8" gracePeriod=30 Jul 24 18:21:03.048824 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.048791 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:03.278349 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.278252 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:03.628887 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.628796 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:03.933101 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.933066 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq"] Jul 24 18:21:03.933501 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.933487 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09af5645-5e0d-462a-a367-7bb84f1c546f" containerName="istio-proxy" Jul 24 18:21:03.933551 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.933505 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="09af5645-5e0d-462a-a367-7bb84f1c546f" containerName="istio-proxy" Jul 24 18:21:03.933551 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.933527 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09af5645-5e0d-462a-a367-7bb84f1c546f" containerName="istio-proxy" Jul 24 18:21:03.933551 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.933536 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="09af5645-5e0d-462a-a367-7bb84f1c546f" containerName="istio-proxy" Jul 24 18:21:03.933642 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.933611 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="09af5645-5e0d-462a-a367-7bb84f1c546f" containerName="istio-proxy" Jul 24 18:21:03.933642 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.933622 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="09af5645-5e0d-462a-a367-7bb84f1c546f" containerName="istio-proxy" Jul 24 18:21:03.937224 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.937200 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:03.939602 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.939580 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-b90928-openshift-default-dockercfg-mdk6k\"" Jul 24 18:21:03.951814 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.951784 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq"] Jul 24 18:21:03.957156 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:03.957128 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:04.074968 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.074894 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-envoy\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.074968 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.074972 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-credential-socket\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.075250 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.075064 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-workload-socket\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.075250 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.075110 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-podinfo\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.075250 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.075127 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/712c7a37-f640-4d7b-996d-c3bab36a349b-istiod-ca-cert\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.075250 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.075158 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4zg7\" (UniqueName: \"kubernetes.io/projected/712c7a37-f640-4d7b-996d-c3bab36a349b-kube-api-access-n4zg7\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.075250 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.075213 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-data\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.075508 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.075267 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-workload-certs\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.075508 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.075330 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-token\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.175878 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.175840 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-workload-socket\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.175878 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.175893 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-podinfo\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.176213 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.175916 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/712c7a37-f640-4d7b-996d-c3bab36a349b-istiod-ca-cert\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.176213 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.175966 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4zg7\" (UniqueName: \"kubernetes.io/projected/712c7a37-f640-4d7b-996d-c3bab36a349b-kube-api-access-n4zg7\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.176213 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.175994 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-data\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.176213 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.176024 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-workload-certs\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.176213 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.176084 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-token\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.176213 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.176128 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-envoy\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.176213 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.176159 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-credential-socket\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.176583 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.176376 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-workload-socket\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.176583 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.176550 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-data\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.176767 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.176740 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-workload-certs\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.176880 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.176789 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-credential-socket\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.176929 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.176876 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/712c7a37-f640-4d7b-996d-c3bab36a349b-istiod-ca-cert\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.178481 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.178460 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-envoy\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.178618 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.178598 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-podinfo\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.184498 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.184435 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-token\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.185175 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.185154 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4zg7\" (UniqueName: \"kubernetes.io/projected/712c7a37-f640-4d7b-996d-c3bab36a349b-kube-api-access-n4zg7\") pod \"e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.244537 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.244503 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:04.247510 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.247490 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:04.392878 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.392853 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq"] Jul 24 18:21:04.395117 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:21:04.395088 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod712c7a37_f640_4d7b_996d_c3bab36a349b.slice/crio-254fb54fa5e45300f135a7e6002532c5e2aa1ca97d63837931de801579a75d6a WatchSource:0}: Error finding container 254fb54fa5e45300f135a7e6002532c5e2aa1ca97d63837931de801579a75d6a: Status 404 returned error can't find the container with id 254fb54fa5e45300f135a7e6002532c5e2aa1ca97d63837931de801579a75d6a Jul 24 18:21:04.397502 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.397472 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:21:04.397577 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.397535 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:21:04.397577 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.397568 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:21:04.864454 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.864367 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" event={"ID":"712c7a37-f640-4d7b-996d-c3bab36a349b","Type":"ContainerStarted","Data":"2a1d0ac368fbe8028a8bba9be80c733d1c63655f6a499b660c65d023973a5725"} Jul 24 18:21:04.864454 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.864425 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" event={"ID":"712c7a37-f640-4d7b-996d-c3bab36a349b","Type":"ContainerStarted","Data":"254fb54fa5e45300f135a7e6002532c5e2aa1ca97d63837931de801579a75d6a"} Jul 24 18:21:04.889195 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:04.889134 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" podStartSLOduration=1.889119353 podStartE2EDuration="1.889119353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:21:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:21:04.887330374 +0000 UTC m=+2460.500336240" watchObservedRunningTime="2026-07-24 18:21:04.889119353 +0000 UTC m=+2460.502125281" Jul 24 18:21:05.248595 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:05.248551 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:05.253656 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:05.253629 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:05.868435 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:05.868400 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:05.869442 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:05.869421 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:06.842292 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:06.842255 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:06.946341 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:06.946305 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:07.080955 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:07.080909 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:07.451783 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:07.451752 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:07.561290 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:07.561259 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:07.648708 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:07.648679 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:08.259487 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.259457 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:21:08.414789 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.414702 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p44pk\" (UniqueName: \"kubernetes.io/projected/ecfb0396-718e-40a0-a4fd-271298f58b15-kube-api-access-p44pk\") pod \"ecfb0396-718e-40a0-a4fd-271298f58b15\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " Jul 24 18:21:08.414789 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.414735 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-token\") pod \"ecfb0396-718e-40a0-a4fd-271298f58b15\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " Jul 24 18:21:08.414789 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.414763 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecfb0396-718e-40a0-a4fd-271298f58b15-istiod-ca-cert\") pod \"ecfb0396-718e-40a0-a4fd-271298f58b15\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " Jul 24 18:21:08.415114 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.414801 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-envoy\") pod \"ecfb0396-718e-40a0-a4fd-271298f58b15\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " Jul 24 18:21:08.415114 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.414826 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-workload-socket\") pod \"ecfb0396-718e-40a0-a4fd-271298f58b15\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " Jul 24 18:21:08.415114 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.414900 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-podinfo\") pod \"ecfb0396-718e-40a0-a4fd-271298f58b15\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " Jul 24 18:21:08.415114 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.415001 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-data\") pod \"ecfb0396-718e-40a0-a4fd-271298f58b15\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " Jul 24 18:21:08.415114 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.415026 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-workload-certs\") pod \"ecfb0396-718e-40a0-a4fd-271298f58b15\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " Jul 24 18:21:08.415114 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.415055 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-credential-socket\") pod \"ecfb0396-718e-40a0-a4fd-271298f58b15\" (UID: \"ecfb0396-718e-40a0-a4fd-271298f58b15\") " Jul 24 18:21:08.415423 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.415209 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ecfb0396-718e-40a0-a4fd-271298f58b15" (UID: "ecfb0396-718e-40a0-a4fd-271298f58b15"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:21:08.415423 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.415323 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ecfb0396-718e-40a0-a4fd-271298f58b15" (UID: "ecfb0396-718e-40a0-a4fd-271298f58b15"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:21:08.415529 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.415429 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:21:08.415529 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.415436 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ecfb0396-718e-40a0-a4fd-271298f58b15" (UID: "ecfb0396-718e-40a0-a4fd-271298f58b15"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:21:08.415529 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.415449 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:21:08.415681 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.415613 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecfb0396-718e-40a0-a4fd-271298f58b15-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ecfb0396-718e-40a0-a4fd-271298f58b15" (UID: "ecfb0396-718e-40a0-a4fd-271298f58b15"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:21:08.416103 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.416074 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-data" (OuterVolumeSpecName: "istio-data") pod "ecfb0396-718e-40a0-a4fd-271298f58b15" (UID: "ecfb0396-718e-40a0-a4fd-271298f58b15"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:21:08.417183 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.417162 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ecfb0396-718e-40a0-a4fd-271298f58b15" (UID: "ecfb0396-718e-40a0-a4fd-271298f58b15"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:21:08.417183 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.417165 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-token" (OuterVolumeSpecName: "istio-token") pod "ecfb0396-718e-40a0-a4fd-271298f58b15" (UID: "ecfb0396-718e-40a0-a4fd-271298f58b15"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:21:08.417314 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.417215 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecfb0396-718e-40a0-a4fd-271298f58b15-kube-api-access-p44pk" (OuterVolumeSpecName: "kube-api-access-p44pk") pod "ecfb0396-718e-40a0-a4fd-271298f58b15" (UID: "ecfb0396-718e-40a0-a4fd-271298f58b15"). InnerVolumeSpecName "kube-api-access-p44pk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:21:08.417352 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.417326 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ecfb0396-718e-40a0-a4fd-271298f58b15" (UID: "ecfb0396-718e-40a0-a4fd-271298f58b15"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:21:08.516683 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.516644 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:21:08.516683 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.516676 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:21:08.516683 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.516686 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:21:08.516925 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.516696 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p44pk\" (UniqueName: \"kubernetes.io/projected/ecfb0396-718e-40a0-a4fd-271298f58b15-kube-api-access-p44pk\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:21:08.516925 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.516706 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:21:08.516925 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.516715 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecfb0396-718e-40a0-a4fd-271298f58b15-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:21:08.516925 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.516722 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecfb0396-718e-40a0-a4fd-271298f58b15-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:21:08.882337 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.882300 2561 generic.go:356] "Generic (PLEG): container finished" podID="ecfb0396-718e-40a0-a4fd-271298f58b15" containerID="e5f3472d9d3931bcf6615488f127316ddf0bb06a6cfc59c165eea0864221d7d8" exitCode=0 Jul 24 18:21:08.882530 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.882353 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" Jul 24 18:21:08.882530 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.882390 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" event={"ID":"ecfb0396-718e-40a0-a4fd-271298f58b15","Type":"ContainerDied","Data":"e5f3472d9d3931bcf6615488f127316ddf0bb06a6cfc59c165eea0864221d7d8"} Jul 24 18:21:08.882530 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.882446 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9" event={"ID":"ecfb0396-718e-40a0-a4fd-271298f58b15","Type":"ContainerDied","Data":"651bb4aa3f7341808e551bac024bbb7ed36092c3496c10c0b9105e0a93d9dca4"} Jul 24 18:21:08.882530 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.882470 2561 scope.go:117] "RemoveContainer" containerID="e5f3472d9d3931bcf6615488f127316ddf0bb06a6cfc59c165eea0864221d7d8" Jul 24 18:21:08.891222 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.891204 2561 scope.go:117] "RemoveContainer" containerID="ff7a1d3c8e44705758573519bcafca2050eb32f53e0b7c2020ca17b1cf28b0f8" Jul 24 18:21:08.899424 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.899406 2561 scope.go:117] "RemoveContainer" containerID="e5f3472d9d3931bcf6615488f127316ddf0bb06a6cfc59c165eea0864221d7d8" Jul 24 18:21:08.899769 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:21:08.899748 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5f3472d9d3931bcf6615488f127316ddf0bb06a6cfc59c165eea0864221d7d8\": container with ID starting with e5f3472d9d3931bcf6615488f127316ddf0bb06a6cfc59c165eea0864221d7d8 not found: ID does not exist" containerID="e5f3472d9d3931bcf6615488f127316ddf0bb06a6cfc59c165eea0864221d7d8" Jul 24 18:21:08.899816 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.899783 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5f3472d9d3931bcf6615488f127316ddf0bb06a6cfc59c165eea0864221d7d8"} err="failed to get container status \"e5f3472d9d3931bcf6615488f127316ddf0bb06a6cfc59c165eea0864221d7d8\": rpc error: code = NotFound desc = could not find container \"e5f3472d9d3931bcf6615488f127316ddf0bb06a6cfc59c165eea0864221d7d8\": container with ID starting with e5f3472d9d3931bcf6615488f127316ddf0bb06a6cfc59c165eea0864221d7d8 not found: ID does not exist" Jul 24 18:21:08.899816 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.899803 2561 scope.go:117] "RemoveContainer" containerID="ff7a1d3c8e44705758573519bcafca2050eb32f53e0b7c2020ca17b1cf28b0f8" Jul 24 18:21:08.900152 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:21:08.900135 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff7a1d3c8e44705758573519bcafca2050eb32f53e0b7c2020ca17b1cf28b0f8\": container with ID starting with ff7a1d3c8e44705758573519bcafca2050eb32f53e0b7c2020ca17b1cf28b0f8 not found: ID does not exist" containerID="ff7a1d3c8e44705758573519bcafca2050eb32f53e0b7c2020ca17b1cf28b0f8" Jul 24 18:21:08.900199 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.900158 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff7a1d3c8e44705758573519bcafca2050eb32f53e0b7c2020ca17b1cf28b0f8"} err="failed to get container status \"ff7a1d3c8e44705758573519bcafca2050eb32f53e0b7c2020ca17b1cf28b0f8\": rpc error: code = NotFound desc = could not find container \"ff7a1d3c8e44705758573519bcafca2050eb32f53e0b7c2020ca17b1cf28b0f8\": container with ID starting with ff7a1d3c8e44705758573519bcafca2050eb32f53e0b7c2020ca17b1cf28b0f8 not found: ID does not exist" Jul 24 18:21:08.908342 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.908309 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9"] Jul 24 18:21:08.914166 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.914135 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-d1a9df-openshift-default-765f9bc69b-lkmc9"] Jul 24 18:21:08.939927 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:08.939899 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecfb0396-718e-40a0-a4fd-271298f58b15" path="/var/lib/kubelet/pods/ecfb0396-718e-40a0-a4fd-271298f58b15/volumes" Jul 24 18:21:11.348056 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:11.348020 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:11.694494 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:11.694462 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:11.985192 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:11.985099 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb"] Jul 24 18:21:11.985536 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:11.985519 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecfb0396-718e-40a0-a4fd-271298f58b15" containerName="istio-proxy" Jul 24 18:21:11.985627 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:11.985536 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecfb0396-718e-40a0-a4fd-271298f58b15" containerName="istio-proxy" Jul 24 18:21:11.985627 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:11.985619 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecfb0396-718e-40a0-a4fd-271298f58b15" containerName="istio-proxy" Jul 24 18:21:11.985742 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:11.985629 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecfb0396-718e-40a0-a4fd-271298f58b15" containerName="istio-proxy" Jul 24 18:21:11.990610 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:11.990585 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:11.992951 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:11.992921 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-88d4a7-openshift-default-dockercfg-qmp9v\"" Jul 24 18:21:12.000962 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.000924 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb"] Jul 24 18:21:12.021522 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.021472 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:12.130628 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.130596 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:12.150203 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.150167 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-workload-certs\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.150386 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.150211 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-data\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.150386 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.150245 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flc62\" (UniqueName: \"kubernetes.io/projected/847ea4af-5170-4d7b-a6dd-fafde1ac0024-kube-api-access-flc62\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.150386 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.150325 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-workload-socket\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.150386 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.150353 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-podinfo\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.150386 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.150382 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-envoy\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.150618 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.150417 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-credential-socket\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.150618 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.150451 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istiod-ca-cert\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.150618 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.150481 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-token\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.251031 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.250903 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-workload-certs\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.251031 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.250997 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-data\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.251264 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.251035 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flc62\" (UniqueName: \"kubernetes.io/projected/847ea4af-5170-4d7b-a6dd-fafde1ac0024-kube-api-access-flc62\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.251264 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.251123 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-workload-socket\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.251264 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.251152 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-podinfo\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.251264 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.251188 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-envoy\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.251264 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.251215 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-credential-socket\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.251531 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.251324 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istiod-ca-cert\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.251531 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.251385 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-workload-certs\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.251531 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.251490 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-data\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.251531 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.251495 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-workload-socket\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.251720 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.251359 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-token\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.251720 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.251660 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-credential-socket\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.252029 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.252003 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istiod-ca-cert\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.253614 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.253578 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-envoy\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.253730 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.253669 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-podinfo\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.258882 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.258850 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-token\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.259026 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.259015 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flc62\" (UniqueName: \"kubernetes.io/projected/847ea4af-5170-4d7b-a6dd-fafde1ac0024-kube-api-access-flc62\") pod \"e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.301671 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.301627 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:12.437653 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.437601 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb"] Jul 24 18:21:12.439747 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:21:12.439718 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod847ea4af_5170_4d7b_a6dd_fafde1ac0024.slice/crio-51125477a46d8469fe40425928335354e9f7d5c1da5537e7965b9dc56106efa9 WatchSource:0}: Error finding container 51125477a46d8469fe40425928335354e9f7d5c1da5537e7965b9dc56106efa9: Status 404 returned error can't find the container with id 51125477a46d8469fe40425928335354e9f7d5c1da5537e7965b9dc56106efa9 Jul 24 18:21:12.441807 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.441773 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:21:12.441905 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.441838 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:21:12.441905 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.441867 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:21:12.901881 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.901840 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" event={"ID":"847ea4af-5170-4d7b-a6dd-fafde1ac0024","Type":"ContainerStarted","Data":"28c4016ae2898761c4937b2e96267135c7d1d3f99605e4fe992abfdecc9caf1c"} Jul 24 18:21:12.902113 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.901896 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" event={"ID":"847ea4af-5170-4d7b-a6dd-fafde1ac0024","Type":"ContainerStarted","Data":"51125477a46d8469fe40425928335354e9f7d5c1da5537e7965b9dc56106efa9"} Jul 24 18:21:12.924894 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:12.924835 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" podStartSLOduration=1.92481779 podStartE2EDuration="1.92481779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:21:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:21:12.923100872 +0000 UTC m=+2468.536106739" watchObservedRunningTime="2026-07-24 18:21:12.92481779 +0000 UTC m=+2468.537823656" Jul 24 18:21:13.302213 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:13.302172 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:13.307102 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:13.307073 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:13.905881 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:13.905839 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:13.906984 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:13.906959 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:14.882128 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:14.882094 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:14.976843 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:14.976807 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:15.689361 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:15.689325 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:16.061852 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:16.061751 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:16.242744 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:16.242714 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:16.327973 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:16.327871 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:19.430859 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:19.430824 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:21:23.949517 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:23.949480 2561 generic.go:356] "Generic (PLEG): container finished" podID="712c7a37-f640-4d7b-996d-c3bab36a349b" containerID="2a1d0ac368fbe8028a8bba9be80c733d1c63655f6a499b660c65d023973a5725" exitCode=0 Jul 24 18:21:23.949982 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:23.949526 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" event={"ID":"712c7a37-f640-4d7b-996d-c3bab36a349b","Type":"ContainerDied","Data":"2a1d0ac368fbe8028a8bba9be80c733d1c63655f6a499b660c65d023973a5725"} Jul 24 18:21:23.949982 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:23.949904 2561 scope.go:117] "RemoveContainer" containerID="2a1d0ac368fbe8028a8bba9be80c733d1c63655f6a499b660c65d023973a5725" Jul 24 18:21:23.950991 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:23.950963 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:21:23.951059 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:23.951028 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:21:23.951105 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:23.951060 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:21:24.956440 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:24.956404 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" event={"ID":"712c7a37-f640-4d7b-996d-c3bab36a349b","Type":"ContainerStarted","Data":"399075ea26ca7aae2004694d481649897c46f01555c3fcf601b14808bec2b057"} Jul 24 18:21:25.248420 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:25.248329 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:25.248614 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:25.248540 2561 patch_prober.go:28] interesting pod/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 24 18:21:25.248686 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:25.248602 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" podUID="712c7a37-f640-4d7b-996d-c3bab36a349b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 24 18:21:26.248653 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:26.248612 2561 patch_prober.go:28] interesting pod/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 24 18:21:26.249083 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:26.248669 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" podUID="712c7a37-f640-4d7b-996d-c3bab36a349b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 24 18:21:27.248789 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:27.248753 2561 patch_prober.go:28] interesting pod/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 24 18:21:27.249236 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:27.248807 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" podUID="712c7a37-f640-4d7b-996d-c3bab36a349b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 24 18:21:28.248702 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:28.248664 2561 patch_prober.go:28] interesting pod/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 24 18:21:28.248886 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:28.248729 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" podUID="712c7a37-f640-4d7b-996d-c3bab36a349b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 24 18:21:29.251526 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:29.251493 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:29.251926 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:29.251804 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:29.252489 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:29.252473 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:21:29.947137 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:29.947096 2561 patch_prober.go:28] interesting pod/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 18:21:29.947314 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:29.947159 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" podUID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 18:21:30.994784 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:30.994698 2561 generic.go:356] "Generic (PLEG): container finished" podID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" containerID="28c4016ae2898761c4937b2e96267135c7d1d3f99605e4fe992abfdecc9caf1c" exitCode=0 Jul 24 18:21:30.994784 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:30.994771 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" event={"ID":"847ea4af-5170-4d7b-a6dd-fafde1ac0024","Type":"ContainerDied","Data":"28c4016ae2898761c4937b2e96267135c7d1d3f99605e4fe992abfdecc9caf1c"} Jul 24 18:21:30.995308 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:30.995295 2561 scope.go:117] "RemoveContainer" containerID="28c4016ae2898761c4937b2e96267135c7d1d3f99605e4fe992abfdecc9caf1c" Jul 24 18:21:30.996332 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:30.996304 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:21:30.996403 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:30.996387 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:21:30.996448 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:30.996434 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:21:32.002101 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:32.002068 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" event={"ID":"847ea4af-5170-4d7b-a6dd-fafde1ac0024","Type":"ContainerStarted","Data":"05a0fb7c60b957bfb69c8a861719a9c8ff39b8ed006135ff76ec5f53d5e868a4"} Jul 24 18:21:32.302783 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:32.302683 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:32.302970 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:32.302903 2561 patch_prober.go:28] interesting pod/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 24 18:21:32.303018 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:32.302981 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" podUID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 24 18:21:33.302202 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:33.302168 2561 patch_prober.go:28] interesting pod/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 24 18:21:33.302577 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:33.302233 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" podUID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 24 18:21:34.303159 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:34.303117 2561 patch_prober.go:28] interesting pod/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 24 18:21:34.303624 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:34.303176 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" podUID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 24 18:21:35.302896 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:35.302852 2561 patch_prober.go:28] interesting pod/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 24 18:21:35.303107 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:35.302922 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" podUID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 24 18:21:36.305976 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:36.305933 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:36.306362 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:36.306232 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:36.306820 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:36.306798 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:21:40.144377 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:21:40.144347 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:22:03.459345 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:22:03.459264 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:22:59.849605 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:22:59.849572 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:22:59.992326 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:22:59.992295 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:23:17.028847 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:23:17.028809 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:24:06.760826 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:24:06.760790 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:24:08.118399 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:24:08.118366 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:24:18.052740 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:24:18.052705 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:24:18.152884 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:24:18.152848 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:24:21.817325 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:24:21.817292 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:24:42.077142 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:24:42.077104 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:24:42.183030 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:24:42.182995 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:24:58.020524 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:24:58.020441 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:25:08.539444 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:25:08.539399 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:25:15.414265 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:25:15.414225 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:25:15.514230 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:25:15.514195 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:25:54.493452 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:25:54.493418 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:25:55.409488 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:25:55.409448 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:26:06.147413 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:26:06.147373 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:26:12.777498 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:26:12.777464 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:26:12.875219 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:26:12.875188 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:26:23.031627 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:26:23.031569 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:15.030037 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:15.030003 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:16.066506 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:16.066466 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:35.103821 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.103784 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:35.396333 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.396237 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9"] Jul 24 18:27:35.396565 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.396552 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecfb0396-718e-40a0-a4fd-271298f58b15" containerName="istio-proxy" Jul 24 18:27:35.396565 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.396565 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecfb0396-718e-40a0-a4fd-271298f58b15" containerName="istio-proxy" Jul 24 18:27:35.399599 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.399574 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.402049 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.402026 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-378627-openshift-default-dockercfg-8klxt\"" Jul 24 18:27:35.413753 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.413720 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9"] Jul 24 18:27:35.423452 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.423422 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:35.427576 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.427540 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-credential-socket\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.427765 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.427596 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-istio-data\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.427765 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.427616 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbd99402-af90-4e07-b922-eae3dccf528f-istio-token\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.427765 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.427632 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-workload-socket\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.427765 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.427710 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-istio-envoy\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.427765 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.427737 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbd99402-af90-4e07-b922-eae3dccf528f-istiod-ca-cert\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.428059 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.427783 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdfjd\" (UniqueName: \"kubernetes.io/projected/cbd99402-af90-4e07-b922-eae3dccf528f-kube-api-access-kdfjd\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.428059 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.427819 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbd99402-af90-4e07-b922-eae3dccf528f-istio-podinfo\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.428059 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.427836 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-workload-certs\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.529288 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.529250 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdfjd\" (UniqueName: \"kubernetes.io/projected/cbd99402-af90-4e07-b922-eae3dccf528f-kube-api-access-kdfjd\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.529288 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.529295 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbd99402-af90-4e07-b922-eae3dccf528f-istio-podinfo\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.529560 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.529315 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-workload-certs\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.529560 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.529344 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-credential-socket\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.529560 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.529372 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-istio-data\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.529560 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.529387 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbd99402-af90-4e07-b922-eae3dccf528f-istio-token\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.529560 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.529404 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-workload-socket\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.529560 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.529463 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-istio-envoy\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.529560 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.529487 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbd99402-af90-4e07-b922-eae3dccf528f-istiod-ca-cert\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.529914 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.529845 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-istio-data\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.529997 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.529917 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-workload-certs\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.529997 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.529973 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-workload-socket\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.530205 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.530181 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbd99402-af90-4e07-b922-eae3dccf528f-istiod-ca-cert\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.530339 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.530301 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-credential-socket\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.531907 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.531873 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-istio-envoy\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.532110 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.532092 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbd99402-af90-4e07-b922-eae3dccf528f-istio-podinfo\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.543044 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.543007 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbd99402-af90-4e07-b922-eae3dccf528f-istio-token\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.543185 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.543100 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdfjd\" (UniqueName: \"kubernetes.io/projected/cbd99402-af90-4e07-b922-eae3dccf528f-kube-api-access-kdfjd\") pod \"e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.712757 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.712714 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:35.722001 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.721921 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:35.858482 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.858453 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9"] Jul 24 18:27:35.860967 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:27:35.860918 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbd99402_af90_4e07_b922_eae3dccf528f.slice/crio-91887a9e7fb948a967c69e7ef276cc9921cb0d96c451c6d00731964fac08712c WatchSource:0}: Error finding container 91887a9e7fb948a967c69e7ef276cc9921cb0d96c451c6d00731964fac08712c: Status 404 returned error can't find the container with id 91887a9e7fb948a967c69e7ef276cc9921cb0d96c451c6d00731964fac08712c Jul 24 18:27:35.862997 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.862969 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:27:35.863062 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.863028 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:27:35.863062 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:35.863059 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:27:36.507600 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:36.507563 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" event={"ID":"cbd99402-af90-4e07-b922-eae3dccf528f","Type":"ContainerStarted","Data":"aea1cdb3cfa9d259fcf0b7aadbb53e3150f50b1ce59bb333dc27116f4059df16"} Jul 24 18:27:36.508058 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:36.507622 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" event={"ID":"cbd99402-af90-4e07-b922-eae3dccf528f","Type":"ContainerStarted","Data":"91887a9e7fb948a967c69e7ef276cc9921cb0d96c451c6d00731964fac08712c"} Jul 24 18:27:36.532974 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:36.532903 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" podStartSLOduration=1.5328835440000002 podStartE2EDuration="1.532883544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:27:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:27:36.53101853 +0000 UTC m=+2852.144024406" watchObservedRunningTime="2026-07-24 18:27:36.532883544 +0000 UTC m=+2852.145889409" Jul 24 18:27:36.713103 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:36.713066 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:36.717974 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:36.717919 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:37.511821 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:37.511787 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:37.512902 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:37.512884 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:38.600650 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:38.600574 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:38.690443 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:38.690388 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:40.021534 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:40.021501 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:40.487289 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:40.487231 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:40.627329 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:40.627296 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:40.738178 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:40.738066 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:43.043590 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:43.043557 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:53.976460 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:53.976402 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:54.429283 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:54.429153 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:54.531171 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:54.531130 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:56.596695 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:56.596662 2561 generic.go:356] "Generic (PLEG): container finished" podID="cbd99402-af90-4e07-b922-eae3dccf528f" containerID="aea1cdb3cfa9d259fcf0b7aadbb53e3150f50b1ce59bb333dc27116f4059df16" exitCode=0 Jul 24 18:27:56.596695 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:56.596701 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" event={"ID":"cbd99402-af90-4e07-b922-eae3dccf528f","Type":"ContainerDied","Data":"aea1cdb3cfa9d259fcf0b7aadbb53e3150f50b1ce59bb333dc27116f4059df16"} Jul 24 18:27:56.597160 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:56.597129 2561 scope.go:117] "RemoveContainer" containerID="aea1cdb3cfa9d259fcf0b7aadbb53e3150f50b1ce59bb333dc27116f4059df16" Jul 24 18:27:56.598178 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:56.598151 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:27:56.598254 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:56.598218 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:27:56.598295 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:56.598251 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:27:57.501895 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:57.501783 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:57.589487 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:57.589449 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:57.602566 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:57.602529 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" event={"ID":"cbd99402-af90-4e07-b922-eae3dccf528f","Type":"ContainerStarted","Data":"276eb134cf99a6b6fea2065043042032eb2ba195adfabe6370c173f443e280e0"} Jul 24 18:27:57.713526 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:57.713477 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:27:57.713802 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:57.713776 2561 patch_prober.go:28] interesting pod/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 24 18:27:57.713878 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:57.713856 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" podUID="cbd99402-af90-4e07-b922-eae3dccf528f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 24 18:27:58.713789 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:58.713744 2561 patch_prober.go:28] interesting pod/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 24 18:27:58.714225 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:58.713810 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" podUID="cbd99402-af90-4e07-b922-eae3dccf528f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 24 18:27:59.090343 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:59.090237 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:59.225200 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:59.225161 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:59.544018 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:59.543924 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:59.641765 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:59.641734 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:27:59.713956 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:59.713895 2561 patch_prober.go:28] interesting pod/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 24 18:27:59.714340 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:27:59.713990 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" podUID="cbd99402-af90-4e07-b922-eae3dccf528f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 24 18:28:00.713629 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:28:00.713581 2561 patch_prober.go:28] interesting pod/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 24 18:28:00.713824 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:28:00.713647 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" podUID="cbd99402-af90-4e07-b922-eae3dccf528f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 24 18:28:01.717374 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:28:01.717340 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:28:01.717784 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:28:01.717715 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:28:01.718511 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:28:01.718491 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:28:02.041973 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:28:02.041864 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:28:36.358559 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:28:36.358519 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:06.976037 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:06.976004 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:07.326746 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.326657 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4"] Jul 24 18:29:07.328816 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.328797 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.332192 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.332171 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-f94d2b-openshift-default-dockercfg-dkqg6\"" Jul 24 18:29:07.338548 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.338519 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:07.346406 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.346371 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4"] Jul 24 18:29:07.371344 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.371304 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-credential-socket\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.371539 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.371352 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-token\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.371539 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.371467 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-workload-certs\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.371539 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.371511 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-data\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.371692 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.371571 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmf8c\" (UniqueName: \"kubernetes.io/projected/601f7982-b374-40c9-aa54-782e9e7bdae5-kube-api-access-vmf8c\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.371692 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.371610 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-workload-socket\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.371692 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.371627 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-podinfo\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.371692 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.371644 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/601f7982-b374-40c9-aa54-782e9e7bdae5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.371692 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.371688 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-envoy\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.472662 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.472625 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-workload-certs\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.472662 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.472662 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-data\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.472922 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.472693 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmf8c\" (UniqueName: \"kubernetes.io/projected/601f7982-b374-40c9-aa54-782e9e7bdae5-kube-api-access-vmf8c\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.472922 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.472712 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-workload-socket\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.472922 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.472729 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-podinfo\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.472922 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.472752 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/601f7982-b374-40c9-aa54-782e9e7bdae5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.472922 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.472778 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-envoy\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.472922 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.472831 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-credential-socket\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.472922 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.472871 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-token\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.473271 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.473205 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-workload-certs\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.473271 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.473215 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-data\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.473500 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.473476 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-credential-socket\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.473601 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.473519 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-workload-socket\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.473601 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.473578 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/601f7982-b374-40c9-aa54-782e9e7bdae5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.475286 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.475262 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-envoy\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.475409 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.475385 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-podinfo\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.481040 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.481008 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmf8c\" (UniqueName: \"kubernetes.io/projected/601f7982-b374-40c9-aa54-782e9e7bdae5-kube-api-access-vmf8c\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.481485 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.481462 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-token\") pod \"e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.497731 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.497697 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:07.639968 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.639856 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:07.774465 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.774437 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4"] Jul 24 18:29:07.776376 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:29:07.776350 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod601f7982_b374_40c9_aa54_782e9e7bdae5.slice/crio-4409d44a94b84be0a1e75ef59bd2da048f449b029f7123d9b1cd9e15a5c30883 WatchSource:0}: Error finding container 4409d44a94b84be0a1e75ef59bd2da048f449b029f7123d9b1cd9e15a5c30883: Status 404 returned error can't find the container with id 4409d44a94b84be0a1e75ef59bd2da048f449b029f7123d9b1cd9e15a5c30883 Jul 24 18:29:07.778663 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.778626 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:29:07.778751 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.778710 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:29:07.778791 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.778749 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:29:07.907360 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.907269 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" event={"ID":"601f7982-b374-40c9-aa54-782e9e7bdae5","Type":"ContainerStarted","Data":"41e96e4f4cd96e30c5b0ce50d4afc21185910404eecef33ffcff943bdef13729"} Jul 24 18:29:07.907360 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.907332 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" event={"ID":"601f7982-b374-40c9-aa54-782e9e7bdae5","Type":"ContainerStarted","Data":"4409d44a94b84be0a1e75ef59bd2da048f449b029f7123d9b1cd9e15a5c30883"} Jul 24 18:29:07.930474 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:07.930414 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" podStartSLOduration=0.930396867 podStartE2EDuration="930.396867ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:29:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:29:07.928092781 +0000 UTC m=+2943.541098646" watchObservedRunningTime="2026-07-24 18:29:07.930396867 +0000 UTC m=+2943.543402733" Jul 24 18:29:08.640438 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:08.640396 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:08.645430 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:08.645403 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:08.911759 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:08.911666 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:08.912546 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:08.912524 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:09.937830 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:09.937792 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:11.080198 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:11.080164 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:12.640477 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:12.640437 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:13.031487 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:13.031439 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:13.130667 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:13.130625 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:13.204922 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:13.204880 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:14.480203 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:14.480172 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:30.565668 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:30.565630 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k"] Jul 24 18:29:30.569324 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:30.569302 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k" Jul 24 18:29:30.571515 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:30.571491 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\"" Jul 24 18:29:30.571641 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:30.571589 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 18:29:30.576690 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:30.576612 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k"] Jul 24 18:29:30.681187 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:30.681152 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af8499e8-a0fd-410a-bc71-f3ad022e7b1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k\" (UID: \"af8499e8-a0fd-410a-bc71-f3ad022e7b1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k" Jul 24 18:29:30.781986 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:30.781917 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af8499e8-a0fd-410a-bc71-f3ad022e7b1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k\" (UID: \"af8499e8-a0fd-410a-bc71-f3ad022e7b1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k" Jul 24 18:29:30.782585 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:30.782558 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af8499e8-a0fd-410a-bc71-f3ad022e7b1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k\" (UID: \"af8499e8-a0fd-410a-bc71-f3ad022e7b1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k" Jul 24 18:29:30.882459 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:30.882364 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k" Jul 24 18:29:31.021183 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:31.021157 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k"] Jul 24 18:29:31.023823 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:29:31.023794 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf8499e8_a0fd_410a_bc71_f3ad022e7b1f.slice/crio-9ce6442e0978f62e0ea75d7e4553c0ff62fdad4eb27ce6e96d36fc3f66500ea1 WatchSource:0}: Error finding container 9ce6442e0978f62e0ea75d7e4553c0ff62fdad4eb27ce6e96d36fc3f66500ea1: Status 404 returned error can't find the container with id 9ce6442e0978f62e0ea75d7e4553c0ff62fdad4eb27ce6e96d36fc3f66500ea1 Jul 24 18:29:32.021358 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:32.021318 2561 generic.go:356] "Generic (PLEG): container finished" podID="af8499e8-a0fd-410a-bc71-f3ad022e7b1f" containerID="368d88fd0e7c25f344e86f13895a157c47b81d563ea3385cf68a9264b480f4a0" exitCode=0 Jul 24 18:29:32.022643 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:32.021408 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k" event={"ID":"af8499e8-a0fd-410a-bc71-f3ad022e7b1f","Type":"ContainerDied","Data":"368d88fd0e7c25f344e86f13895a157c47b81d563ea3385cf68a9264b480f4a0"} Jul 24 18:29:32.022643 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:32.021481 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k" event={"ID":"af8499e8-a0fd-410a-bc71-f3ad022e7b1f","Type":"ContainerStarted","Data":"9ce6442e0978f62e0ea75d7e4553c0ff62fdad4eb27ce6e96d36fc3f66500ea1"} Jul 24 18:29:33.027255 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:33.027218 2561 generic.go:356] "Generic (PLEG): container finished" podID="601f7982-b374-40c9-aa54-782e9e7bdae5" containerID="41e96e4f4cd96e30c5b0ce50d4afc21185910404eecef33ffcff943bdef13729" exitCode=0 Jul 24 18:29:33.027668 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:33.027287 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" event={"ID":"601f7982-b374-40c9-aa54-782e9e7bdae5","Type":"ContainerDied","Data":"41e96e4f4cd96e30c5b0ce50d4afc21185910404eecef33ffcff943bdef13729"} Jul 24 18:29:33.028014 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:33.027829 2561 scope.go:117] "RemoveContainer" containerID="41e96e4f4cd96e30c5b0ce50d4afc21185910404eecef33ffcff943bdef13729" Jul 24 18:29:33.028926 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:33.028893 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:29:33.029062 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:33.028997 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:29:33.029062 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:33.029047 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:29:33.183208 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:33.183173 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k" Jul 24 18:29:33.303316 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:33.303218 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af8499e8-a0fd-410a-bc71-f3ad022e7b1f-maas-api-service-ca\") pod \"af8499e8-a0fd-410a-bc71-f3ad022e7b1f\" (UID: \"af8499e8-a0fd-410a-bc71-f3ad022e7b1f\") " Jul 24 18:29:33.303640 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:33.303609 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af8499e8-a0fd-410a-bc71-f3ad022e7b1f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "af8499e8-a0fd-410a-bc71-f3ad022e7b1f" (UID: "af8499e8-a0fd-410a-bc71-f3ad022e7b1f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:29:33.404026 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:33.403983 2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af8499e8-a0fd-410a-bc71-f3ad022e7b1f-maas-api-service-ca\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:29:34.033542 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:34.033507 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" event={"ID":"601f7982-b374-40c9-aa54-782e9e7bdae5","Type":"ContainerStarted","Data":"3cc2318f47fe4150a3186702bc9c9a84b06b5b182a708902ea3c004ee49d5b7f"} Jul 24 18:29:34.034757 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:34.034734 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k" event={"ID":"af8499e8-a0fd-410a-bc71-f3ad022e7b1f","Type":"ContainerDied","Data":"9ce6442e0978f62e0ea75d7e4553c0ff62fdad4eb27ce6e96d36fc3f66500ea1"} Jul 24 18:29:34.034859 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:34.034759 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k" Jul 24 18:29:34.034859 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:34.034774 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ce6442e0978f62e0ea75d7e4553c0ff62fdad4eb27ce6e96d36fc3f66500ea1" Jul 24 18:29:34.640357 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:34.640319 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:34.641374 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:34.641350 2561 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 24 18:29:34.641491 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:34.641399 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" podUID="601f7982-b374-40c9-aa54-782e9e7bdae5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 24 18:29:35.640465 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:35.640425 2561 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 24 18:29:35.640996 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:35.640491 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" podUID="601f7982-b374-40c9-aa54-782e9e7bdae5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 24 18:29:36.640444 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:36.640403 2561 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 24 18:29:36.640634 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:36.640472 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" podUID="601f7982-b374-40c9-aa54-782e9e7bdae5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 24 18:29:37.640495 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:37.640463 2561 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 24 18:29:37.640707 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:37.640527 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" podUID="601f7982-b374-40c9-aa54-782e9e7bdae5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 24 18:29:38.644599 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:38.644569 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:38.645029 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:38.644830 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:38.645661 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:38.645645 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:41.987895 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:41.987851 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:42.088049 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:42.088011 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:43.450097 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:43.450062 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:43.548876 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:43.548841 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:43.648138 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:43.648107 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:45.643759 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:45.643676 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k"] Jul 24 18:29:45.643759 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:45.643736 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:45.648931 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:45.648908 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f94d2b-45w6k"] Jul 24 18:29:46.035510 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:46.035474 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:46.940640 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:46.940602 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af8499e8-a0fd-410a-bc71-f3ad022e7b1f" path="/var/lib/kubelet/pods/af8499e8-a0fd-410a-bc71-f3ad022e7b1f/volumes" Jul 24 18:29:52.003984 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:52.003954 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:52.104114 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:52.104078 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:52.319608 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:52.319522 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4"] Jul 24 18:29:52.319826 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:52.319799 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" podUID="601f7982-b374-40c9-aa54-782e9e7bdae5" containerName="istio-proxy" containerID="cri-o://3cc2318f47fe4150a3186702bc9c9a84b06b5b182a708902ea3c004ee49d5b7f" gracePeriod=30 Jul 24 18:29:52.334236 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:52.334206 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:52.523670 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:52.523635 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:29:53.647505 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:53.647466 2561 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 18:29:53.647904 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:53.647541 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" podUID="601f7982-b374-40c9-aa54-782e9e7bdae5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 18:29:57.571441 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.571413 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:57.722489 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.722445 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-workload-socket\") pod \"601f7982-b374-40c9-aa54-782e9e7bdae5\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " Jul 24 18:29:57.722489 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.722494 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmf8c\" (UniqueName: \"kubernetes.io/projected/601f7982-b374-40c9-aa54-782e9e7bdae5-kube-api-access-vmf8c\") pod \"601f7982-b374-40c9-aa54-782e9e7bdae5\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " Jul 24 18:29:57.722770 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.722514 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/601f7982-b374-40c9-aa54-782e9e7bdae5-istiod-ca-cert\") pod \"601f7982-b374-40c9-aa54-782e9e7bdae5\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " Jul 24 18:29:57.722770 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.722545 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-workload-certs\") pod \"601f7982-b374-40c9-aa54-782e9e7bdae5\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " Jul 24 18:29:57.722770 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.722564 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-token\") pod \"601f7982-b374-40c9-aa54-782e9e7bdae5\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " Jul 24 18:29:57.722770 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.722591 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-podinfo\") pod \"601f7982-b374-40c9-aa54-782e9e7bdae5\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " Jul 24 18:29:57.722770 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.722632 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-data\") pod \"601f7982-b374-40c9-aa54-782e9e7bdae5\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " Jul 24 18:29:57.722770 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.722652 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-credential-socket\") pod \"601f7982-b374-40c9-aa54-782e9e7bdae5\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " Jul 24 18:29:57.722770 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.722725 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-envoy\") pod \"601f7982-b374-40c9-aa54-782e9e7bdae5\" (UID: \"601f7982-b374-40c9-aa54-782e9e7bdae5\") " Jul 24 18:29:57.723163 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.722873 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "601f7982-b374-40c9-aa54-782e9e7bdae5" (UID: "601f7982-b374-40c9-aa54-782e9e7bdae5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:29:57.723163 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.723009 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/601f7982-b374-40c9-aa54-782e9e7bdae5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "601f7982-b374-40c9-aa54-782e9e7bdae5" (UID: "601f7982-b374-40c9-aa54-782e9e7bdae5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:29:57.723163 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.722880 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "601f7982-b374-40c9-aa54-782e9e7bdae5" (UID: "601f7982-b374-40c9-aa54-782e9e7bdae5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:29:57.723163 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.723097 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:29:57.723368 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.723275 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "601f7982-b374-40c9-aa54-782e9e7bdae5" (UID: "601f7982-b374-40c9-aa54-782e9e7bdae5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:29:57.723684 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.723663 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-data" (OuterVolumeSpecName: "istio-data") pod "601f7982-b374-40c9-aa54-782e9e7bdae5" (UID: "601f7982-b374-40c9-aa54-782e9e7bdae5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:29:57.724847 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.724816 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-token" (OuterVolumeSpecName: "istio-token") pod "601f7982-b374-40c9-aa54-782e9e7bdae5" (UID: "601f7982-b374-40c9-aa54-782e9e7bdae5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:29:57.725031 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.724872 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/601f7982-b374-40c9-aa54-782e9e7bdae5-kube-api-access-vmf8c" (OuterVolumeSpecName: "kube-api-access-vmf8c") pod "601f7982-b374-40c9-aa54-782e9e7bdae5" (UID: "601f7982-b374-40c9-aa54-782e9e7bdae5"). InnerVolumeSpecName "kube-api-access-vmf8c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:29:57.725171 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.725151 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "601f7982-b374-40c9-aa54-782e9e7bdae5" (UID: "601f7982-b374-40c9-aa54-782e9e7bdae5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:29:57.725230 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.725201 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "601f7982-b374-40c9-aa54-782e9e7bdae5" (UID: "601f7982-b374-40c9-aa54-782e9e7bdae5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:29:57.823770 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.823733 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:29:57.823770 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.823766 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmf8c\" (UniqueName: \"kubernetes.io/projected/601f7982-b374-40c9-aa54-782e9e7bdae5-kube-api-access-vmf8c\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:29:57.823770 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.823779 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/601f7982-b374-40c9-aa54-782e9e7bdae5-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:29:57.824079 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.823788 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:29:57.824079 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.823796 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:29:57.824079 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.823803 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:29:57.824079 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.823811 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:29:57.824079 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:57.823819 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/601f7982-b374-40c9-aa54-782e9e7bdae5-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:29:58.141972 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:58.141871 2561 generic.go:356] "Generic (PLEG): container finished" podID="601f7982-b374-40c9-aa54-782e9e7bdae5" containerID="3cc2318f47fe4150a3186702bc9c9a84b06b5b182a708902ea3c004ee49d5b7f" exitCode=0 Jul 24 18:29:58.141972 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:58.141911 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" event={"ID":"601f7982-b374-40c9-aa54-782e9e7bdae5","Type":"ContainerDied","Data":"3cc2318f47fe4150a3186702bc9c9a84b06b5b182a708902ea3c004ee49d5b7f"} Jul 24 18:29:58.141972 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:58.141933 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" Jul 24 18:29:58.141972 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:58.141968 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4" event={"ID":"601f7982-b374-40c9-aa54-782e9e7bdae5","Type":"ContainerDied","Data":"4409d44a94b84be0a1e75ef59bd2da048f449b029f7123d9b1cd9e15a5c30883"} Jul 24 18:29:58.142233 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:58.141990 2561 scope.go:117] "RemoveContainer" containerID="3cc2318f47fe4150a3186702bc9c9a84b06b5b182a708902ea3c004ee49d5b7f" Jul 24 18:29:58.151041 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:58.151021 2561 scope.go:117] "RemoveContainer" containerID="41e96e4f4cd96e30c5b0ce50d4afc21185910404eecef33ffcff943bdef13729" Jul 24 18:29:58.159025 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:58.159000 2561 scope.go:117] "RemoveContainer" containerID="3cc2318f47fe4150a3186702bc9c9a84b06b5b182a708902ea3c004ee49d5b7f" Jul 24 18:29:58.159288 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:29:58.159266 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cc2318f47fe4150a3186702bc9c9a84b06b5b182a708902ea3c004ee49d5b7f\": container with ID starting with 3cc2318f47fe4150a3186702bc9c9a84b06b5b182a708902ea3c004ee49d5b7f not found: ID does not exist" containerID="3cc2318f47fe4150a3186702bc9c9a84b06b5b182a708902ea3c004ee49d5b7f" Jul 24 18:29:58.159336 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:58.159297 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cc2318f47fe4150a3186702bc9c9a84b06b5b182a708902ea3c004ee49d5b7f"} err="failed to get container status \"3cc2318f47fe4150a3186702bc9c9a84b06b5b182a708902ea3c004ee49d5b7f\": rpc error: code = NotFound desc = could not find container \"3cc2318f47fe4150a3186702bc9c9a84b06b5b182a708902ea3c004ee49d5b7f\": container with ID starting with 3cc2318f47fe4150a3186702bc9c9a84b06b5b182a708902ea3c004ee49d5b7f not found: ID does not exist" Jul 24 18:29:58.159336 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:58.159315 2561 scope.go:117] "RemoveContainer" containerID="41e96e4f4cd96e30c5b0ce50d4afc21185910404eecef33ffcff943bdef13729" Jul 24 18:29:58.159553 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:29:58.159536 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41e96e4f4cd96e30c5b0ce50d4afc21185910404eecef33ffcff943bdef13729\": container with ID starting with 41e96e4f4cd96e30c5b0ce50d4afc21185910404eecef33ffcff943bdef13729 not found: ID does not exist" containerID="41e96e4f4cd96e30c5b0ce50d4afc21185910404eecef33ffcff943bdef13729" Jul 24 18:29:58.159610 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:58.159562 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41e96e4f4cd96e30c5b0ce50d4afc21185910404eecef33ffcff943bdef13729"} err="failed to get container status \"41e96e4f4cd96e30c5b0ce50d4afc21185910404eecef33ffcff943bdef13729\": rpc error: code = NotFound desc = could not find container \"41e96e4f4cd96e30c5b0ce50d4afc21185910404eecef33ffcff943bdef13729\": container with ID starting with 41e96e4f4cd96e30c5b0ce50d4afc21185910404eecef33ffcff943bdef13729 not found: ID does not exist" Jul 24 18:29:58.163384 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:58.163357 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4"] Jul 24 18:29:58.167847 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:58.167826 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f94d2b-openshift-default-5c6785f6db-9wpn4"] Jul 24 18:29:58.940344 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:29:58.940306 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="601f7982-b374-40c9-aa54-782e9e7bdae5" path="/var/lib/kubelet/pods/601f7982-b374-40c9-aa54-782e9e7bdae5/volumes" Jul 24 18:30:00.145710 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.145670 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b90928-29748630-jp4rd"] Jul 24 18:30:00.146204 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.146148 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="601f7982-b374-40c9-aa54-782e9e7bdae5" containerName="istio-proxy" Jul 24 18:30:00.146204 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.146165 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="601f7982-b374-40c9-aa54-782e9e7bdae5" containerName="istio-proxy" Jul 24 18:30:00.146204 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.146180 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af8499e8-a0fd-410a-bc71-f3ad022e7b1f" containerName="revoke-keys" Jul 24 18:30:00.146204 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.146189 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="af8499e8-a0fd-410a-bc71-f3ad022e7b1f" containerName="revoke-keys" Jul 24 18:30:00.146334 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.146281 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="af8499e8-a0fd-410a-bc71-f3ad022e7b1f" containerName="revoke-keys" Jul 24 18:30:00.146334 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.146293 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="601f7982-b374-40c9-aa54-782e9e7bdae5" containerName="istio-proxy" Jul 24 18:30:00.149234 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.149208 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-378627-29748630-qwz9b"] Jul 24 18:30:00.149416 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.149331 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b90928-29748630-jp4rd" Jul 24 18:30:00.149638 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.149617 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="601f7982-b374-40c9-aa54-782e9e7bdae5" containerName="istio-proxy" Jul 24 18:30:00.149728 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.149638 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="601f7982-b374-40c9-aa54-782e9e7bdae5" containerName="istio-proxy" Jul 24 18:30:00.149793 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.149730 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="601f7982-b374-40c9-aa54-782e9e7bdae5" containerName="istio-proxy" Jul 24 18:30:00.151711 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.151685 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-88d4a7-29748630-jh68v"] Jul 24 18:30:00.151891 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.151746 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hglbt\"" Jul 24 18:30:00.151891 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.151815 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-378627-29748630-qwz9b" Jul 24 18:30:00.153672 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.153638 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-88d4a7-29748630-jh68v" Jul 24 18:30:00.159428 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.159403 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b90928-29748630-jp4rd"] Jul 24 18:30:00.161824 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.161793 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-88d4a7-29748630-jh68v"] Jul 24 18:30:00.162405 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.162382 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b90928-29748630-jp4rd" Jul 24 18:30:00.164039 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.164016 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-378627-29748630-qwz9b"] Jul 24 18:30:00.190010 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.189743 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-378627-29748630-qwz9b" Jul 24 18:30:00.198775 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.198628 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-88d4a7-29748630-jh68v" Jul 24 18:30:00.331607 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.331573 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b90928-29748630-jp4rd"] Jul 24 18:30:00.334064 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:30:00.334007 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6547e34_d82d_44ce_9fd9_b7690e0379b2.slice/crio-701f1e32a691c8aead1f11972b6ba0032db3806228b2b29f81d8392a92a3914a WatchSource:0}: Error finding container 701f1e32a691c8aead1f11972b6ba0032db3806228b2b29f81d8392a92a3914a: Status 404 returned error can't find the container with id 701f1e32a691c8aead1f11972b6ba0032db3806228b2b29f81d8392a92a3914a Jul 24 18:30:00.384171 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.384120 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-88d4a7-29748630-jh68v"] Jul 24 18:30:00.387700 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:30:00.387671 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda31fe8b4_7773_4a06_aa2d_2ac946bd9449.slice/crio-a10a8d78dd8bc72f9a68b74cf2967c573f4a1b5c2410c04733e266f12cfaf073 WatchSource:0}: Error finding container a10a8d78dd8bc72f9a68b74cf2967c573f4a1b5c2410c04733e266f12cfaf073: Status 404 returned error can't find the container with id a10a8d78dd8bc72f9a68b74cf2967c573f4a1b5c2410c04733e266f12cfaf073 Jul 24 18:30:00.564482 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:00.564319 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-378627-29748630-qwz9b"] Jul 24 18:30:01.159187 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:01.159087 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-88d4a7-29748630-jh68v" event={"ID":"a31fe8b4-7773-4a06-aa2d-2ac946bd9449","Type":"ContainerStarted","Data":"f71273430a4eec05894c8f04e41e7cb6889532efdde2ecbfc0f13be52d77cce6"} Jul 24 18:30:01.159187 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:01.159150 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-88d4a7-29748630-jh68v" event={"ID":"a31fe8b4-7773-4a06-aa2d-2ac946bd9449","Type":"ContainerStarted","Data":"a10a8d78dd8bc72f9a68b74cf2967c573f4a1b5c2410c04733e266f12cfaf073"} Jul 24 18:30:01.160567 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:01.160539 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b90928-29748630-jp4rd" event={"ID":"c6547e34-d82d-44ce-9fd9-b7690e0379b2","Type":"ContainerStarted","Data":"d3e4f13135af37908cfd5188608a1eb7d803de5c37a0e4c24e03051731b0c008"} Jul 24 18:30:01.160686 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:01.160584 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b90928-29748630-jp4rd" event={"ID":"c6547e34-d82d-44ce-9fd9-b7690e0379b2","Type":"ContainerStarted","Data":"701f1e32a691c8aead1f11972b6ba0032db3806228b2b29f81d8392a92a3914a"} Jul 24 18:30:01.161952 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:01.161917 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-378627-29748630-qwz9b" event={"ID":"592acf40-b768-4e33-868f-48f9c9d1d29f","Type":"ContainerStarted","Data":"daf833a72a0e7e1d4d3dbff62660ce3bb2eaab06d959ea55bca7503d80d70cc6"} Jul 24 18:30:01.162035 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:01.161974 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-378627-29748630-qwz9b" event={"ID":"592acf40-b768-4e33-868f-48f9c9d1d29f","Type":"ContainerStarted","Data":"326624a903d157fc2101ac50fbab3b5e6eee168013a3e1563042ab831c44044f"} Jul 24 18:30:01.175466 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:01.175398 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-88d4a7-29748630-jh68v" podStartSLOduration=1.175378304 podStartE2EDuration="1.175378304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:30:01.174309341 +0000 UTC m=+2996.787315240" watchObservedRunningTime="2026-07-24 18:30:01.175378304 +0000 UTC m=+2996.788384168" Jul 24 18:30:01.189295 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:01.189245 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b90928-29748630-jp4rd" podStartSLOduration=1.189228836 podStartE2EDuration="1.189228836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:30:01.187914906 +0000 UTC m=+2996.800920773" watchObservedRunningTime="2026-07-24 18:30:01.189228836 +0000 UTC m=+2996.802234701" Jul 24 18:30:01.201455 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:01.201406 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-378627-29748630-qwz9b" podStartSLOduration=1.201391016 podStartE2EDuration="1.201391016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:30:01.200348693 +0000 UTC m=+2996.813354559" watchObservedRunningTime="2026-07-24 18:30:01.201391016 +0000 UTC m=+2996.814396882" Jul 24 18:30:02.167995 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:02.167964 2561 generic.go:356] "Generic (PLEG): container finished" podID="a31fe8b4-7773-4a06-aa2d-2ac946bd9449" containerID="f71273430a4eec05894c8f04e41e7cb6889532efdde2ecbfc0f13be52d77cce6" exitCode=0 Jul 24 18:30:02.168410 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:02.168031 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-88d4a7-29748630-jh68v" event={"ID":"a31fe8b4-7773-4a06-aa2d-2ac946bd9449","Type":"ContainerDied","Data":"f71273430a4eec05894c8f04e41e7cb6889532efdde2ecbfc0f13be52d77cce6"} Jul 24 18:30:02.169451 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:02.169426 2561 generic.go:356] "Generic (PLEG): container finished" podID="c6547e34-d82d-44ce-9fd9-b7690e0379b2" containerID="d3e4f13135af37908cfd5188608a1eb7d803de5c37a0e4c24e03051731b0c008" exitCode=0 Jul 24 18:30:02.169554 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:02.169504 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b90928-29748630-jp4rd" event={"ID":"c6547e34-d82d-44ce-9fd9-b7690e0379b2","Type":"ContainerDied","Data":"d3e4f13135af37908cfd5188608a1eb7d803de5c37a0e4c24e03051731b0c008"} Jul 24 18:30:02.170819 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:02.170798 2561 generic.go:356] "Generic (PLEG): container finished" podID="592acf40-b768-4e33-868f-48f9c9d1d29f" containerID="daf833a72a0e7e1d4d3dbff62660ce3bb2eaab06d959ea55bca7503d80d70cc6" exitCode=0 Jul 24 18:30:02.170950 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:02.170821 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-378627-29748630-qwz9b" event={"ID":"592acf40-b768-4e33-868f-48f9c9d1d29f","Type":"ContainerDied","Data":"daf833a72a0e7e1d4d3dbff62660ce3bb2eaab06d959ea55bca7503d80d70cc6"} Jul 24 18:30:03.314876 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:03.314832 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b90928-29748630-jp4rd" Jul 24 18:30:03.366841 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:03.366814 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-88d4a7-29748630-jh68v" Jul 24 18:30:03.370077 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:03.370055 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-378627-29748630-qwz9b" Jul 24 18:30:04.180619 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:04.180582 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-378627-29748630-qwz9b" Jul 24 18:30:04.180619 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:04.180608 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-378627-29748630-qwz9b" event={"ID":"592acf40-b768-4e33-868f-48f9c9d1d29f","Type":"ContainerDied","Data":"326624a903d157fc2101ac50fbab3b5e6eee168013a3e1563042ab831c44044f"} Jul 24 18:30:04.180855 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:04.180661 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="326624a903d157fc2101ac50fbab3b5e6eee168013a3e1563042ab831c44044f" Jul 24 18:30:04.182052 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:04.182024 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-88d4a7-29748630-jh68v" Jul 24 18:30:04.182179 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:04.182025 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-88d4a7-29748630-jh68v" event={"ID":"a31fe8b4-7773-4a06-aa2d-2ac946bd9449","Type":"ContainerDied","Data":"a10a8d78dd8bc72f9a68b74cf2967c573f4a1b5c2410c04733e266f12cfaf073"} Jul 24 18:30:04.182179 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:04.182136 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a10a8d78dd8bc72f9a68b74cf2967c573f4a1b5c2410c04733e266f12cfaf073" Jul 24 18:30:04.183381 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:04.183355 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b90928-29748630-jp4rd" event={"ID":"c6547e34-d82d-44ce-9fd9-b7690e0379b2","Type":"ContainerDied","Data":"701f1e32a691c8aead1f11972b6ba0032db3806228b2b29f81d8392a92a3914a"} Jul 24 18:30:04.183520 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:04.183381 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b90928-29748630-jp4rd" Jul 24 18:30:04.183520 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:04.183393 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="701f1e32a691c8aead1f11972b6ba0032db3806228b2b29f81d8392a92a3914a" Jul 24 18:30:14.947508 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:14.947474 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-378627-29748630-qwz9b"] Jul 24 18:30:14.949370 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:14.949345 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-378627-29748630-qwz9b"] Jul 24 18:30:14.950596 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:14.950568 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:30:15.022511 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:15.022455 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:30:15.083510 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:15.083476 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:30:16.195729 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:16.195701 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:30:16.647856 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:16.647769 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:30:16.946329 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:16.946291 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="592acf40-b768-4e33-868f-48f9c9d1d29f" path="/var/lib/kubelet/pods/592acf40-b768-4e33-868f-48f9c9d1d29f/volumes" Jul 24 18:30:21.686262 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:21.686230 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:30:30.295213 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:30.295178 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:30:30.397473 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:30.397441 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:30:59.091256 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:59.091172 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:30:59.212076 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:30:59.212026 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:05.046362 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:05.046329 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:10.910710 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:10.910668 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:11.006278 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:11.006243 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:11.245670 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:11.245634 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9"] Jul 24 18:31:11.245900 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:11.245874 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" podUID="cbd99402-af90-4e07-b922-eae3dccf528f" containerName="istio-proxy" containerID="cri-o://276eb134cf99a6b6fea2065043042032eb2ba195adfabe6370c173f443e280e0" gracePeriod=30 Jul 24 18:31:11.294450 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:11.294397 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:11.470422 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:11.470390 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:16.502573 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.502543 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:31:16.503075 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.503051 2561 generic.go:356] "Generic (PLEG): container finished" podID="cbd99402-af90-4e07-b922-eae3dccf528f" containerID="276eb134cf99a6b6fea2065043042032eb2ba195adfabe6370c173f443e280e0" exitCode=0 Jul 24 18:31:16.503153 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.503110 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" event={"ID":"cbd99402-af90-4e07-b922-eae3dccf528f","Type":"ContainerDied","Data":"276eb134cf99a6b6fea2065043042032eb2ba195adfabe6370c173f443e280e0"} Jul 24 18:31:16.503224 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.503167 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" event={"ID":"cbd99402-af90-4e07-b922-eae3dccf528f","Type":"ContainerDied","Data":"91887a9e7fb948a967c69e7ef276cc9921cb0d96c451c6d00731964fac08712c"} Jul 24 18:31:16.503224 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.503201 2561 scope.go:117] "RemoveContainer" containerID="276eb134cf99a6b6fea2065043042032eb2ba195adfabe6370c173f443e280e0" Jul 24 18:31:16.511524 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.511505 2561 scope.go:117] "RemoveContainer" containerID="aea1cdb3cfa9d259fcf0b7aadbb53e3150f50b1ce59bb333dc27116f4059df16" Jul 24 18:31:16.519765 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.519743 2561 scope.go:117] "RemoveContainer" containerID="276eb134cf99a6b6fea2065043042032eb2ba195adfabe6370c173f443e280e0" Jul 24 18:31:16.520040 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:31:16.520022 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"276eb134cf99a6b6fea2065043042032eb2ba195adfabe6370c173f443e280e0\": container with ID starting with 276eb134cf99a6b6fea2065043042032eb2ba195adfabe6370c173f443e280e0 not found: ID does not exist" containerID="276eb134cf99a6b6fea2065043042032eb2ba195adfabe6370c173f443e280e0" Jul 24 18:31:16.520091 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.520050 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"276eb134cf99a6b6fea2065043042032eb2ba195adfabe6370c173f443e280e0"} err="failed to get container status \"276eb134cf99a6b6fea2065043042032eb2ba195adfabe6370c173f443e280e0\": rpc error: code = NotFound desc = could not find container \"276eb134cf99a6b6fea2065043042032eb2ba195adfabe6370c173f443e280e0\": container with ID starting with 276eb134cf99a6b6fea2065043042032eb2ba195adfabe6370c173f443e280e0 not found: ID does not exist" Jul 24 18:31:16.520091 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.520076 2561 scope.go:117] "RemoveContainer" containerID="aea1cdb3cfa9d259fcf0b7aadbb53e3150f50b1ce59bb333dc27116f4059df16" Jul 24 18:31:16.520328 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:31:16.520305 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aea1cdb3cfa9d259fcf0b7aadbb53e3150f50b1ce59bb333dc27116f4059df16\": container with ID starting with aea1cdb3cfa9d259fcf0b7aadbb53e3150f50b1ce59bb333dc27116f4059df16 not found: ID does not exist" containerID="aea1cdb3cfa9d259fcf0b7aadbb53e3150f50b1ce59bb333dc27116f4059df16" Jul 24 18:31:16.520368 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.520337 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aea1cdb3cfa9d259fcf0b7aadbb53e3150f50b1ce59bb333dc27116f4059df16"} err="failed to get container status \"aea1cdb3cfa9d259fcf0b7aadbb53e3150f50b1ce59bb333dc27116f4059df16\": rpc error: code = NotFound desc = could not find container \"aea1cdb3cfa9d259fcf0b7aadbb53e3150f50b1ce59bb333dc27116f4059df16\": container with ID starting with aea1cdb3cfa9d259fcf0b7aadbb53e3150f50b1ce59bb333dc27116f4059df16 not found: ID does not exist" Jul 24 18:31:16.618918 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.618880 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-workload-socket\") pod \"cbd99402-af90-4e07-b922-eae3dccf528f\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " Jul 24 18:31:16.619151 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.618930 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-istio-envoy\") pod \"cbd99402-af90-4e07-b922-eae3dccf528f\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " Jul 24 18:31:16.619151 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.618978 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-workload-certs\") pod \"cbd99402-af90-4e07-b922-eae3dccf528f\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " Jul 24 18:31:16.619151 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.618999 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-istio-data\") pod \"cbd99402-af90-4e07-b922-eae3dccf528f\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " Jul 24 18:31:16.619151 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.619039 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdfjd\" (UniqueName: \"kubernetes.io/projected/cbd99402-af90-4e07-b922-eae3dccf528f-kube-api-access-kdfjd\") pod \"cbd99402-af90-4e07-b922-eae3dccf528f\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " Jul 24 18:31:16.619151 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.619076 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbd99402-af90-4e07-b922-eae3dccf528f-istiod-ca-cert\") pod \"cbd99402-af90-4e07-b922-eae3dccf528f\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " Jul 24 18:31:16.619151 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.619117 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbd99402-af90-4e07-b922-eae3dccf528f-istio-token\") pod \"cbd99402-af90-4e07-b922-eae3dccf528f\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " Jul 24 18:31:16.619151 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.619146 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-credential-socket\") pod \"cbd99402-af90-4e07-b922-eae3dccf528f\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " Jul 24 18:31:16.619526 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.619169 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbd99402-af90-4e07-b922-eae3dccf528f-istio-podinfo\") pod \"cbd99402-af90-4e07-b922-eae3dccf528f\" (UID: \"cbd99402-af90-4e07-b922-eae3dccf528f\") " Jul 24 18:31:16.619526 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.619224 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cbd99402-af90-4e07-b922-eae3dccf528f" (UID: "cbd99402-af90-4e07-b922-eae3dccf528f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:31:16.619526 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.619291 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cbd99402-af90-4e07-b922-eae3dccf528f" (UID: "cbd99402-af90-4e07-b922-eae3dccf528f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:31:16.619526 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.619524 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:31:16.619710 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.619542 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:31:16.619752 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.619725 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbd99402-af90-4e07-b922-eae3dccf528f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cbd99402-af90-4e07-b922-eae3dccf528f" (UID: "cbd99402-af90-4e07-b922-eae3dccf528f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:31:16.619840 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.619790 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cbd99402-af90-4e07-b922-eae3dccf528f" (UID: "cbd99402-af90-4e07-b922-eae3dccf528f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:31:16.620204 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.620178 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-istio-data" (OuterVolumeSpecName: "istio-data") pod "cbd99402-af90-4e07-b922-eae3dccf528f" (UID: "cbd99402-af90-4e07-b922-eae3dccf528f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:31:16.621330 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.621308 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cbd99402-af90-4e07-b922-eae3dccf528f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cbd99402-af90-4e07-b922-eae3dccf528f" (UID: "cbd99402-af90-4e07-b922-eae3dccf528f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:31:16.621436 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.621341 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbd99402-af90-4e07-b922-eae3dccf528f-kube-api-access-kdfjd" (OuterVolumeSpecName: "kube-api-access-kdfjd") pod "cbd99402-af90-4e07-b922-eae3dccf528f" (UID: "cbd99402-af90-4e07-b922-eae3dccf528f"). InnerVolumeSpecName "kube-api-access-kdfjd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:31:16.621590 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.621563 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbd99402-af90-4e07-b922-eae3dccf528f-istio-token" (OuterVolumeSpecName: "istio-token") pod "cbd99402-af90-4e07-b922-eae3dccf528f" (UID: "cbd99402-af90-4e07-b922-eae3dccf528f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:31:16.621590 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.621577 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cbd99402-af90-4e07-b922-eae3dccf528f" (UID: "cbd99402-af90-4e07-b922-eae3dccf528f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:31:16.720782 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.720749 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:31:16.720782 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.720779 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdfjd\" (UniqueName: \"kubernetes.io/projected/cbd99402-af90-4e07-b922-eae3dccf528f-kube-api-access-kdfjd\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:31:16.720982 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.720790 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbd99402-af90-4e07-b922-eae3dccf528f-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:31:16.720982 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.720798 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbd99402-af90-4e07-b922-eae3dccf528f-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:31:16.720982 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.720806 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:31:16.720982 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.720814 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbd99402-af90-4e07-b922-eae3dccf528f-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:31:16.720982 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:16.720821 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbd99402-af90-4e07-b922-eae3dccf528f-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:31:17.509305 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:17.509272 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9" Jul 24 18:31:17.526175 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:17.526142 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9"] Jul 24 18:31:17.530345 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:17.530317 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-378627-openshift-default-6fc4545bcb-wghq9"] Jul 24 18:31:18.940895 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:18.940861 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbd99402-af90-4e07-b922-eae3dccf528f" path="/var/lib/kubelet/pods/cbd99402-af90-4e07-b922-eae3dccf528f/volumes" Jul 24 18:31:24.233520 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:24.233478 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:24.345501 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:24.345469 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:26.762337 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:26.762301 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:27.223733 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:27.223699 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:30.999579 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:30.999547 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:31.068308 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:31.068277 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:33.670673 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:33.670629 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:33.771044 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:33.770998 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:33.968242 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:33.968209 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:34.265422 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:34.265328 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:46.653704 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:46.653667 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b90928-29748630-jp4rd"] Jul 24 18:31:46.656904 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:46.656871 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b90928-29748630-jp4rd"] Jul 24 18:31:46.662178 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:46.662151 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:46.728354 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:46.728317 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:46.787995 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:46.787924 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:46.940720 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:46.940684 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6547e34-d82d-44ce-9fd9-b7690e0379b2" path="/var/lib/kubelet/pods/c6547e34-d82d-44ce-9fd9-b7690e0379b2/volumes" Jul 24 18:31:49.607757 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:49.607724 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:50.001015 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:50.000985 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:56.490165 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:56.490123 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:56.596234 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:56.596195 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:56.764200 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:56.764116 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq"] Jul 24 18:31:56.764390 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:56.764363 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" podUID="712c7a37-f640-4d7b-996d-c3bab36a349b" containerName="istio-proxy" containerID="cri-o://399075ea26ca7aae2004694d481649897c46f01555c3fcf601b14808bec2b057" gracePeriod=30 Jul 24 18:31:56.823755 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:56.823702 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:57.021685 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:57.021595 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:31:59.254273 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:59.254233 2561 patch_prober.go:28] interesting pod/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 18:31:59.254645 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:31:59.254295 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" podUID="712c7a37-f640-4d7b-996d-c3bab36a349b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 18:32:00.915401 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:00.915363 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:32:02.013679 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.013654 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:32:02.108630 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.108540 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-envoy\") pod \"712c7a37-f640-4d7b-996d-c3bab36a349b\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " Jul 24 18:32:02.108630 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.108588 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-podinfo\") pod \"712c7a37-f640-4d7b-996d-c3bab36a349b\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " Jul 24 18:32:02.108872 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.108641 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-credential-socket\") pod \"712c7a37-f640-4d7b-996d-c3bab36a349b\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " Jul 24 18:32:02.108872 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.108658 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-data\") pod \"712c7a37-f640-4d7b-996d-c3bab36a349b\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " Jul 24 18:32:02.108872 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.108677 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-token\") pod \"712c7a37-f640-4d7b-996d-c3bab36a349b\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " Jul 24 18:32:02.108872 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.108706 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4zg7\" (UniqueName: \"kubernetes.io/projected/712c7a37-f640-4d7b-996d-c3bab36a349b-kube-api-access-n4zg7\") pod \"712c7a37-f640-4d7b-996d-c3bab36a349b\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " Jul 24 18:32:02.108872 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.108734 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/712c7a37-f640-4d7b-996d-c3bab36a349b-istiod-ca-cert\") pod \"712c7a37-f640-4d7b-996d-c3bab36a349b\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " Jul 24 18:32:02.108872 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.108764 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-workload-socket\") pod \"712c7a37-f640-4d7b-996d-c3bab36a349b\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " Jul 24 18:32:02.108872 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.108805 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-workload-certs\") pod \"712c7a37-f640-4d7b-996d-c3bab36a349b\" (UID: \"712c7a37-f640-4d7b-996d-c3bab36a349b\") " Jul 24 18:32:02.109267 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.109028 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "712c7a37-f640-4d7b-996d-c3bab36a349b" (UID: "712c7a37-f640-4d7b-996d-c3bab36a349b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:32:02.109267 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.109250 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "712c7a37-f640-4d7b-996d-c3bab36a349b" (UID: "712c7a37-f640-4d7b-996d-c3bab36a349b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:32:02.109488 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.109460 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/712c7a37-f640-4d7b-996d-c3bab36a349b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "712c7a37-f640-4d7b-996d-c3bab36a349b" (UID: "712c7a37-f640-4d7b-996d-c3bab36a349b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:32:02.109560 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.109539 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "712c7a37-f640-4d7b-996d-c3bab36a349b" (UID: "712c7a37-f640-4d7b-996d-c3bab36a349b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:32:02.109732 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.109701 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-data" (OuterVolumeSpecName: "istio-data") pod "712c7a37-f640-4d7b-996d-c3bab36a349b" (UID: "712c7a37-f640-4d7b-996d-c3bab36a349b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:32:02.110898 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.110875 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-token" (OuterVolumeSpecName: "istio-token") pod "712c7a37-f640-4d7b-996d-c3bab36a349b" (UID: "712c7a37-f640-4d7b-996d-c3bab36a349b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:32:02.110997 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.110978 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "712c7a37-f640-4d7b-996d-c3bab36a349b" (UID: "712c7a37-f640-4d7b-996d-c3bab36a349b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:32:02.111261 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.111242 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/712c7a37-f640-4d7b-996d-c3bab36a349b-kube-api-access-n4zg7" (OuterVolumeSpecName: "kube-api-access-n4zg7") pod "712c7a37-f640-4d7b-996d-c3bab36a349b" (UID: "712c7a37-f640-4d7b-996d-c3bab36a349b"). InnerVolumeSpecName "kube-api-access-n4zg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:32:02.111306 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.111252 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "712c7a37-f640-4d7b-996d-c3bab36a349b" (UID: "712c7a37-f640-4d7b-996d-c3bab36a349b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:32:02.209855 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.209813 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:02.209855 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.209850 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:02.209855 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.209862 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:02.210140 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.209873 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4zg7\" (UniqueName: \"kubernetes.io/projected/712c7a37-f640-4d7b-996d-c3bab36a349b-kube-api-access-n4zg7\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:02.210140 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.209885 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/712c7a37-f640-4d7b-996d-c3bab36a349b-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:02.210140 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.209895 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:02.210140 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.209908 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:02.210140 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.209918 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:02.210140 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.209929 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/712c7a37-f640-4d7b-996d-c3bab36a349b-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:02.702991 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.702951 2561 generic.go:356] "Generic (PLEG): container finished" podID="712c7a37-f640-4d7b-996d-c3bab36a349b" containerID="399075ea26ca7aae2004694d481649897c46f01555c3fcf601b14808bec2b057" exitCode=0 Jul 24 18:32:02.703189 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.702999 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" Jul 24 18:32:02.703189 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.703044 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" event={"ID":"712c7a37-f640-4d7b-996d-c3bab36a349b","Type":"ContainerDied","Data":"399075ea26ca7aae2004694d481649897c46f01555c3fcf601b14808bec2b057"} Jul 24 18:32:02.703189 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.703099 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq" event={"ID":"712c7a37-f640-4d7b-996d-c3bab36a349b","Type":"ContainerDied","Data":"254fb54fa5e45300f135a7e6002532c5e2aa1ca97d63837931de801579a75d6a"} Jul 24 18:32:02.703189 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.703122 2561 scope.go:117] "RemoveContainer" containerID="399075ea26ca7aae2004694d481649897c46f01555c3fcf601b14808bec2b057" Jul 24 18:32:02.712333 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.712314 2561 scope.go:117] "RemoveContainer" containerID="2a1d0ac368fbe8028a8bba9be80c733d1c63655f6a499b660c65d023973a5725" Jul 24 18:32:02.721458 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.721437 2561 scope.go:117] "RemoveContainer" containerID="399075ea26ca7aae2004694d481649897c46f01555c3fcf601b14808bec2b057" Jul 24 18:32:02.721774 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:32:02.721755 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"399075ea26ca7aae2004694d481649897c46f01555c3fcf601b14808bec2b057\": container with ID starting with 399075ea26ca7aae2004694d481649897c46f01555c3fcf601b14808bec2b057 not found: ID does not exist" containerID="399075ea26ca7aae2004694d481649897c46f01555c3fcf601b14808bec2b057" Jul 24 18:32:02.721848 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.721783 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"399075ea26ca7aae2004694d481649897c46f01555c3fcf601b14808bec2b057"} err="failed to get container status \"399075ea26ca7aae2004694d481649897c46f01555c3fcf601b14808bec2b057\": rpc error: code = NotFound desc = could not find container \"399075ea26ca7aae2004694d481649897c46f01555c3fcf601b14808bec2b057\": container with ID starting with 399075ea26ca7aae2004694d481649897c46f01555c3fcf601b14808bec2b057 not found: ID does not exist" Jul 24 18:32:02.721848 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.721799 2561 scope.go:117] "RemoveContainer" containerID="2a1d0ac368fbe8028a8bba9be80c733d1c63655f6a499b660c65d023973a5725" Jul 24 18:32:02.722082 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:32:02.722064 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a1d0ac368fbe8028a8bba9be80c733d1c63655f6a499b660c65d023973a5725\": container with ID starting with 2a1d0ac368fbe8028a8bba9be80c733d1c63655f6a499b660c65d023973a5725 not found: ID does not exist" containerID="2a1d0ac368fbe8028a8bba9be80c733d1c63655f6a499b660c65d023973a5725" Jul 24 18:32:02.722135 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.722088 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a1d0ac368fbe8028a8bba9be80c733d1c63655f6a499b660c65d023973a5725"} err="failed to get container status \"2a1d0ac368fbe8028a8bba9be80c733d1c63655f6a499b660c65d023973a5725\": rpc error: code = NotFound desc = could not find container \"2a1d0ac368fbe8028a8bba9be80c733d1c63655f6a499b660c65d023973a5725\": container with ID starting with 2a1d0ac368fbe8028a8bba9be80c733d1c63655f6a499b660c65d023973a5725 not found: ID does not exist" Jul 24 18:32:02.726979 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.726956 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq"] Jul 24 18:32:02.731376 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.731353 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-b90928-openshift-default-59df98d5bd-2bptq"] Jul 24 18:32:02.940305 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:02.940275 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="712c7a37-f640-4d7b-996d-c3bab36a349b" path="/var/lib/kubelet/pods/712c7a37-f640-4d7b-996d-c3bab36a349b/volumes" Jul 24 18:32:08.176630 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:08.176592 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-88d4a7-29748630-jh68v"] Jul 24 18:32:08.179420 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:08.179391 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-88d4a7-29748630-jh68v"] Jul 24 18:32:08.181977 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:08.181953 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:32:08.282848 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:08.282816 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:32:08.384358 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:08.384327 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:32:08.941070 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:08.941034 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a31fe8b4-7773-4a06-aa2d-2ac946bd9449" path="/var/lib/kubelet/pods/a31fe8b4-7773-4a06-aa2d-2ac946bd9449/volumes" Jul 24 18:32:12.270538 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:12.270499 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:32:12.668489 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:12.668400 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:32:18.891534 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:18.891500 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:32:18.994798 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:18.994765 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:32:19.176650 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:19.176571 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb"] Jul 24 18:32:19.176814 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:19.176789 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" podUID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" containerName="istio-proxy" containerID="cri-o://05a0fb7c60b957bfb69c8a861719a9c8ff39b8ed006135ff76ec5f53d5e868a4" gracePeriod=30 Jul 24 18:32:19.209276 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:19.209236 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:32:19.436998 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:19.436963 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:32:21.308954 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:21.308900 2561 patch_prober.go:28] interesting pod/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 18:32:21.309321 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:21.308991 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" podUID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 18:32:24.425792 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.425768 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:32:24.509832 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.509753 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-workload-certs\") pod \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " Jul 24 18:32:24.509832 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.509790 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flc62\" (UniqueName: \"kubernetes.io/projected/847ea4af-5170-4d7b-a6dd-fafde1ac0024-kube-api-access-flc62\") pod \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " Jul 24 18:32:24.509832 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.509814 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-envoy\") pod \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " Jul 24 18:32:24.510094 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.509931 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-data\") pod \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " Jul 24 18:32:24.510094 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.509988 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-credential-socket\") pod \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " Jul 24 18:32:24.510094 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.510032 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istiod-ca-cert\") pod \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " Jul 24 18:32:24.510094 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.510060 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "847ea4af-5170-4d7b-a6dd-fafde1ac0024" (UID: "847ea4af-5170-4d7b-a6dd-fafde1ac0024"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:32:24.510094 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.510081 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-podinfo\") pod \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " Jul 24 18:32:24.510366 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.510128 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-workload-socket\") pod \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " Jul 24 18:32:24.510366 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.510155 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-token\") pod \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\" (UID: \"847ea4af-5170-4d7b-a6dd-fafde1ac0024\") " Jul 24 18:32:24.510366 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.510276 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "847ea4af-5170-4d7b-a6dd-fafde1ac0024" (UID: "847ea4af-5170-4d7b-a6dd-fafde1ac0024"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:32:24.510514 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.510374 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "847ea4af-5170-4d7b-a6dd-fafde1ac0024" (UID: "847ea4af-5170-4d7b-a6dd-fafde1ac0024"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:32:24.510514 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.510452 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:24.510514 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.510472 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:24.510514 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.510487 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:24.510705 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.510647 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "847ea4af-5170-4d7b-a6dd-fafde1ac0024" (UID: "847ea4af-5170-4d7b-a6dd-fafde1ac0024"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:32:24.511022 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.510996 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-data" (OuterVolumeSpecName: "istio-data") pod "847ea4af-5170-4d7b-a6dd-fafde1ac0024" (UID: "847ea4af-5170-4d7b-a6dd-fafde1ac0024"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:32:24.512113 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.512087 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/847ea4af-5170-4d7b-a6dd-fafde1ac0024-kube-api-access-flc62" (OuterVolumeSpecName: "kube-api-access-flc62") pod "847ea4af-5170-4d7b-a6dd-fafde1ac0024" (UID: "847ea4af-5170-4d7b-a6dd-fafde1ac0024"). InnerVolumeSpecName "kube-api-access-flc62". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:32:24.512228 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.512147 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "847ea4af-5170-4d7b-a6dd-fafde1ac0024" (UID: "847ea4af-5170-4d7b-a6dd-fafde1ac0024"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:32:24.512369 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.512347 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "847ea4af-5170-4d7b-a6dd-fafde1ac0024" (UID: "847ea4af-5170-4d7b-a6dd-fafde1ac0024"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:32:24.512521 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.512502 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-token" (OuterVolumeSpecName: "istio-token") pod "847ea4af-5170-4d7b-a6dd-fafde1ac0024" (UID: "847ea4af-5170-4d7b-a6dd-fafde1ac0024"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:32:24.611731 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.611700 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flc62\" (UniqueName: \"kubernetes.io/projected/847ea4af-5170-4d7b-a6dd-fafde1ac0024-kube-api-access-flc62\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:24.611731 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.611727 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:24.611731 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.611737 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:24.611961 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.611746 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:24.611961 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.611754 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/847ea4af-5170-4d7b-a6dd-fafde1ac0024-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:24.611961 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.611763 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/847ea4af-5170-4d7b-a6dd-fafde1ac0024-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 24 18:32:24.796367 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.796290 2561 generic.go:356] "Generic (PLEG): container finished" podID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" containerID="05a0fb7c60b957bfb69c8a861719a9c8ff39b8ed006135ff76ec5f53d5e868a4" exitCode=0 Jul 24 18:32:24.796367 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.796338 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" Jul 24 18:32:24.796541 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.796376 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" event={"ID":"847ea4af-5170-4d7b-a6dd-fafde1ac0024","Type":"ContainerDied","Data":"05a0fb7c60b957bfb69c8a861719a9c8ff39b8ed006135ff76ec5f53d5e868a4"} Jul 24 18:32:24.796541 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.796434 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb" event={"ID":"847ea4af-5170-4d7b-a6dd-fafde1ac0024","Type":"ContainerDied","Data":"51125477a46d8469fe40425928335354e9f7d5c1da5537e7965b9dc56106efa9"} Jul 24 18:32:24.796541 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.796463 2561 scope.go:117] "RemoveContainer" containerID="05a0fb7c60b957bfb69c8a861719a9c8ff39b8ed006135ff76ec5f53d5e868a4" Jul 24 18:32:24.805104 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.805087 2561 scope.go:117] "RemoveContainer" containerID="28c4016ae2898761c4937b2e96267135c7d1d3f99605e4fe992abfdecc9caf1c" Jul 24 18:32:24.812555 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.812537 2561 scope.go:117] "RemoveContainer" containerID="05a0fb7c60b957bfb69c8a861719a9c8ff39b8ed006135ff76ec5f53d5e868a4" Jul 24 18:32:24.812783 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:32:24.812762 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05a0fb7c60b957bfb69c8a861719a9c8ff39b8ed006135ff76ec5f53d5e868a4\": container with ID starting with 05a0fb7c60b957bfb69c8a861719a9c8ff39b8ed006135ff76ec5f53d5e868a4 not found: ID does not exist" containerID="05a0fb7c60b957bfb69c8a861719a9c8ff39b8ed006135ff76ec5f53d5e868a4" Jul 24 18:32:24.812830 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.812791 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05a0fb7c60b957bfb69c8a861719a9c8ff39b8ed006135ff76ec5f53d5e868a4"} err="failed to get container status \"05a0fb7c60b957bfb69c8a861719a9c8ff39b8ed006135ff76ec5f53d5e868a4\": rpc error: code = NotFound desc = could not find container \"05a0fb7c60b957bfb69c8a861719a9c8ff39b8ed006135ff76ec5f53d5e868a4\": container with ID starting with 05a0fb7c60b957bfb69c8a861719a9c8ff39b8ed006135ff76ec5f53d5e868a4 not found: ID does not exist" Jul 24 18:32:24.812830 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.812807 2561 scope.go:117] "RemoveContainer" containerID="28c4016ae2898761c4937b2e96267135c7d1d3f99605e4fe992abfdecc9caf1c" Jul 24 18:32:24.813066 ip-10-0-137-185 kubenswrapper[2561]: E0724 18:32:24.813044 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28c4016ae2898761c4937b2e96267135c7d1d3f99605e4fe992abfdecc9caf1c\": container with ID starting with 28c4016ae2898761c4937b2e96267135c7d1d3f99605e4fe992abfdecc9caf1c not found: ID does not exist" containerID="28c4016ae2898761c4937b2e96267135c7d1d3f99605e4fe992abfdecc9caf1c" Jul 24 18:32:24.813163 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.813069 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28c4016ae2898761c4937b2e96267135c7d1d3f99605e4fe992abfdecc9caf1c"} err="failed to get container status \"28c4016ae2898761c4937b2e96267135c7d1d3f99605e4fe992abfdecc9caf1c\": rpc error: code = NotFound desc = could not find container \"28c4016ae2898761c4937b2e96267135c7d1d3f99605e4fe992abfdecc9caf1c\": container with ID starting with 28c4016ae2898761c4937b2e96267135c7d1d3f99605e4fe992abfdecc9caf1c not found: ID does not exist" Jul 24 18:32:24.817886 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.817865 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb"] Jul 24 18:32:24.822065 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.822045 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-88d4a7-openshift-default-7c7ddbcf76-k85fb"] Jul 24 18:32:24.944905 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:24.944874 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" path="/var/lib/kubelet/pods/847ea4af-5170-4d7b-a6dd-fafde1ac0024/volumes" Jul 24 18:32:26.368006 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:26.367968 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f579866-4zsfh_3bb64cee-06b6-47c6-b80b-72f990f7b77e/copy-manifests/0.log" Jul 24 18:32:26.374969 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:26.374927 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f579866-4zsfh_3bb64cee-06b6-47c6-b80b-72f990f7b77e/manager/0.log" Jul 24 18:32:27.153873 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:27.153792 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748600-gms8p_886ddf63-c505-41e1-989d-994c58a56bea/cleanup/0.log" Jul 24 18:32:30.344900 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:30.344867 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c_7197fed7-f249-49db-8de0-851291502d13/istio-proxy/0.log" Jul 24 18:32:30.487858 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:30.487830 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-lxrvw_9ddbbb6d-d271-498f-bb27-2259d6b4e9b7/discovery/0.log" Jul 24 18:32:31.756508 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:31.756478 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-mpxcq_33e860bf-aba3-43a3-a09e-dffcdb381294/main/0.log" Jul 24 18:32:31.881549 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:31.881516 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hczg9_3089e9ab-532b-496f-b595-1a670a39a96f/main/0.log" Jul 24 18:32:32.126302 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:32.126222 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-8zszk_a7fb3593-eead-458d-a105-ad939a220cad/main/0.log" Jul 24 18:32:38.047717 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:38.047639 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:32:38.145792 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:38.145762 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" limit=1024 Jul 24 18:32:40.905918 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:40.905866 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zp4x2_a3c435cc-32ae-4849-a167-4cf7d91b6832/global-pull-secret-syncer/0.log" Jul 24 18:32:40.955766 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:40.955739 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5lrd6_25eb2f6d-27ac-4259-bc89-fbcaa6cf085e/konnectivity-agent/0.log" Jul 24 18:32:41.068115 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:41.068070 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-185.ec2.internal_824071a194219c7cb41a1892ae483511/haproxy/0.log" Jul 24 18:32:48.933913 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:48.933880 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vfw2t_25543d3e-6225-4815-92aa-237ee763a248/node-exporter/0.log" Jul 24 18:32:48.956757 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:48.956732 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vfw2t_25543d3e-6225-4815-92aa-237ee763a248/kube-rbac-proxy/0.log" Jul 24 18:32:48.980733 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:48.980707 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vfw2t_25543d3e-6225-4815-92aa-237ee763a248/init-textfile/0.log" Jul 24 18:32:51.212169 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212132 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh"] Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212420 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="712c7a37-f640-4d7b-996d-c3bab36a349b" containerName="istio-proxy" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212435 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="712c7a37-f640-4d7b-996d-c3bab36a349b" containerName="istio-proxy" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212445 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a31fe8b4-7773-4a06-aa2d-2ac946bd9449" containerName="cleanup" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212451 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a31fe8b4-7773-4a06-aa2d-2ac946bd9449" containerName="cleanup" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212460 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" containerName="istio-proxy" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212466 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" containerName="istio-proxy" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212474 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6547e34-d82d-44ce-9fd9-b7690e0379b2" containerName="cleanup" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212478 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6547e34-d82d-44ce-9fd9-b7690e0379b2" containerName="cleanup" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212488 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="592acf40-b768-4e33-868f-48f9c9d1d29f" containerName="cleanup" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212493 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="592acf40-b768-4e33-868f-48f9c9d1d29f" containerName="cleanup" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212504 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbd99402-af90-4e07-b922-eae3dccf528f" containerName="istio-proxy" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212513 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbd99402-af90-4e07-b922-eae3dccf528f" containerName="istio-proxy" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212522 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="712c7a37-f640-4d7b-996d-c3bab36a349b" containerName="istio-proxy" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212527 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="712c7a37-f640-4d7b-996d-c3bab36a349b" containerName="istio-proxy" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212534 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbd99402-af90-4e07-b922-eae3dccf528f" containerName="istio-proxy" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212539 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbd99402-af90-4e07-b922-eae3dccf528f" containerName="istio-proxy" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212546 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" containerName="istio-proxy" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212551 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" containerName="istio-proxy" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212601 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbd99402-af90-4e07-b922-eae3dccf528f" containerName="istio-proxy" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212608 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6547e34-d82d-44ce-9fd9-b7690e0379b2" containerName="cleanup" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212615 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" containerName="istio-proxy" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212620 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="712c7a37-f640-4d7b-996d-c3bab36a349b" containerName="istio-proxy" Jul 24 18:32:51.212616 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212627 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a31fe8b4-7773-4a06-aa2d-2ac946bd9449" containerName="cleanup" Jul 24 18:32:51.213328 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212631 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="592acf40-b768-4e33-868f-48f9c9d1d29f" containerName="cleanup" Jul 24 18:32:51.213328 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.212637 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="847ea4af-5170-4d7b-a6dd-fafde1ac0024" containerName="istio-proxy" Jul 24 18:32:51.215482 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.215466 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.217677 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.217655 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dhhl7\"/\"kube-root-ca.crt\"" Jul 24 18:32:51.217817 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.217739 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dhhl7\"/\"openshift-service-ca.crt\"" Jul 24 18:32:51.218045 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.218028 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dhhl7\"/\"default-dockercfg-hrkj5\"" Jul 24 18:32:51.225487 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.225463 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh"] Jul 24 18:32:51.333363 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.333330 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d0fa15af-2208-4be9-9d84-ab7e04ffe5b8-podres\") pod \"perf-node-gather-daemonset-pvmjh\" (UID: \"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.333533 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.333377 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27ccf\" (UniqueName: \"kubernetes.io/projected/d0fa15af-2208-4be9-9d84-ab7e04ffe5b8-kube-api-access-27ccf\") pod \"perf-node-gather-daemonset-pvmjh\" (UID: \"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.333533 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.333419 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d0fa15af-2208-4be9-9d84-ab7e04ffe5b8-lib-modules\") pod \"perf-node-gather-daemonset-pvmjh\" (UID: \"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.333533 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.333439 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d0fa15af-2208-4be9-9d84-ab7e04ffe5b8-proc\") pod \"perf-node-gather-daemonset-pvmjh\" (UID: \"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.333533 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.333459 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d0fa15af-2208-4be9-9d84-ab7e04ffe5b8-sys\") pod \"perf-node-gather-daemonset-pvmjh\" (UID: \"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.434571 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.434537 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d0fa15af-2208-4be9-9d84-ab7e04ffe5b8-podres\") pod \"perf-node-gather-daemonset-pvmjh\" (UID: \"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.434759 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.434586 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27ccf\" (UniqueName: \"kubernetes.io/projected/d0fa15af-2208-4be9-9d84-ab7e04ffe5b8-kube-api-access-27ccf\") pod \"perf-node-gather-daemonset-pvmjh\" (UID: \"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.434759 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.434635 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d0fa15af-2208-4be9-9d84-ab7e04ffe5b8-lib-modules\") pod \"perf-node-gather-daemonset-pvmjh\" (UID: \"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.434759 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.434666 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d0fa15af-2208-4be9-9d84-ab7e04ffe5b8-proc\") pod \"perf-node-gather-daemonset-pvmjh\" (UID: \"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.434759 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.434691 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d0fa15af-2208-4be9-9d84-ab7e04ffe5b8-sys\") pod \"perf-node-gather-daemonset-pvmjh\" (UID: \"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.434759 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.434734 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d0fa15af-2208-4be9-9d84-ab7e04ffe5b8-podres\") pod \"perf-node-gather-daemonset-pvmjh\" (UID: \"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.434997 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.434764 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d0fa15af-2208-4be9-9d84-ab7e04ffe5b8-sys\") pod \"perf-node-gather-daemonset-pvmjh\" (UID: \"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.434997 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.434792 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d0fa15af-2208-4be9-9d84-ab7e04ffe5b8-proc\") pod \"perf-node-gather-daemonset-pvmjh\" (UID: \"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.434997 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.434817 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d0fa15af-2208-4be9-9d84-ab7e04ffe5b8-lib-modules\") pod \"perf-node-gather-daemonset-pvmjh\" (UID: \"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.443208 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.443186 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27ccf\" (UniqueName: \"kubernetes.io/projected/d0fa15af-2208-4be9-9d84-ab7e04ffe5b8-kube-api-access-27ccf\") pod \"perf-node-gather-daemonset-pvmjh\" (UID: \"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8\") " pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.525641 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.525561 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.649719 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.649635 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh"] Jul 24 18:32:51.652491 ip-10-0-137-185 kubenswrapper[2561]: W0724 18:32:51.652460 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd0fa15af_2208_4be9_9d84_ab7e04ffe5b8.slice/crio-418da768a8bc75d272b7fe8015b8c9a215b08da559fd150098458fd770506e94 WatchSource:0}: Error finding container 418da768a8bc75d272b7fe8015b8c9a215b08da559fd150098458fd770506e94: Status 404 returned error can't find the container with id 418da768a8bc75d272b7fe8015b8c9a215b08da559fd150098458fd770506e94 Jul 24 18:32:51.912400 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.912363 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" event={"ID":"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8","Type":"ContainerStarted","Data":"41e57908ae195e0839576a3dcb5173b21756b434196b2f70a4146517ed857caf"} Jul 24 18:32:51.912562 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.912415 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" event={"ID":"d0fa15af-2208-4be9-9d84-ab7e04ffe5b8","Type":"ContainerStarted","Data":"418da768a8bc75d272b7fe8015b8c9a215b08da559fd150098458fd770506e94"} Jul 24 18:32:51.912562 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.912454 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:32:51.929648 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:51.929601 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" podStartSLOduration=0.929585696 podStartE2EDuration="929.585696ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:32:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:32:51.928512679 +0000 UTC m=+3167.541518558" watchObservedRunningTime="2026-07-24 18:32:51.929585696 +0000 UTC m=+3167.542591599" Jul 24 18:32:53.789900 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:53.789866 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tmsv8_21676a3c-bd55-4e4a-ab8e-4275108751f2/dns/0.log" Jul 24 18:32:53.811838 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:53.811809 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tmsv8_21676a3c-bd55-4e4a-ab8e-4275108751f2/kube-rbac-proxy/0.log" Jul 24 18:32:53.860407 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:53.860380 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rdlc9_e6d525b8-1f5d-4568-ae50-d71eb372ca67/dns-node-resolver/0.log" Jul 24 18:32:54.414233 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:54.414202 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-794b4ccc5-mtfft_00ae222f-d539-492e-bbbe-5973669634e9/registry/0.log" Jul 24 18:32:54.437465 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:54.437439 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2r8wb_525c91a9-d541-474e-9c6a-90cd9d3c460d/node-ca/0.log" Jul 24 18:32:55.425283 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:55.425253 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c_7197fed7-f249-49db-8de0-851291502d13/istio-proxy/0.log" Jul 24 18:32:55.574123 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:55.574089 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-lxrvw_9ddbbb6d-d271-498f-bb27-2259d6b4e9b7/discovery/0.log" Jul 24 18:32:56.614422 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:56.614395 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lxj4f_e12eb736-035d-481f-afcf-9a09118b554a/serve-healthcheck-canary/0.log" Jul 24 18:32:57.296108 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:57.296081 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zcvr7_afa66580-b192-471f-a0a1-2eadd6d85817/kube-rbac-proxy/0.log" Jul 24 18:32:57.316036 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:57.316008 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zcvr7_afa66580-b192-471f-a0a1-2eadd6d85817/exporter/0.log" Jul 24 18:32:57.337257 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:57.337233 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zcvr7_afa66580-b192-471f-a0a1-2eadd6d85817/extractor/0.log" Jul 24 18:32:57.926457 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:32:57.926432 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dhhl7/perf-node-gather-daemonset-pvmjh" Jul 24 18:33:00.046174 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:00.046144 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f579866-4zsfh_3bb64cee-06b6-47c6-b80b-72f990f7b77e/manager/0.log" Jul 24 18:33:00.066892 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:00.066867 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f579866-4zsfh_3bb64cee-06b6-47c6-b80b-72f990f7b77e/copy-manifests/0.log" Jul 24 18:33:01.591641 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:01.591611 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c_7197fed7-f249-49db-8de0-851291502d13/istio-proxy/0.log" Jul 24 18:33:01.702132 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:01.702097 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-74c56c9c54-x99lt_1f0a75bb-350c-4cff-b6ae-b6266320aec3/manager/0.log" Jul 24 18:33:01.726573 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:01.726497 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-lxrvw_9ddbbb6d-d271-498f-bb27-2259d6b4e9b7/discovery/0.log" Jul 24 18:33:03.243915 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:03.243881 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-7d67c_7197fed7-f249-49db-8de0-851291502d13/istio-proxy/0.log" Jul 24 18:33:03.382594 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:03.382556 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-lxrvw_9ddbbb6d-d271-498f-bb27-2259d6b4e9b7/discovery/0.log" Jul 24 18:33:06.647028 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:06.646997 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-lpw5h_4944123d-acba-498e-9140-a1645675f417/migrator/0.log" Jul 24 18:33:06.667886 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:06.667857 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-lpw5h_4944123d-acba-498e-9140-a1645675f417/graceful-termination/0.log" Jul 24 18:33:08.087116 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:08.087086 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-72fk4_5200e3bb-ae3f-4798-9909-0657616b6c7a/kube-multus/0.log" Jul 24 18:33:08.267699 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:08.267673 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fzfj6_397fd802-aa1f-4a3e-bd2b-6835089bbf23/kube-multus-additional-cni-plugins/0.log" Jul 24 18:33:08.288284 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:08.288258 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fzfj6_397fd802-aa1f-4a3e-bd2b-6835089bbf23/egress-router-binary-copy/0.log" Jul 24 18:33:08.308148 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:08.308120 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fzfj6_397fd802-aa1f-4a3e-bd2b-6835089bbf23/cni-plugins/0.log" Jul 24 18:33:08.327163 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:08.327144 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fzfj6_397fd802-aa1f-4a3e-bd2b-6835089bbf23/bond-cni-plugin/0.log" Jul 24 18:33:08.346700 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:08.346644 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fzfj6_397fd802-aa1f-4a3e-bd2b-6835089bbf23/routeoverride-cni/0.log" Jul 24 18:33:08.365373 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:08.365353 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fzfj6_397fd802-aa1f-4a3e-bd2b-6835089bbf23/whereabouts-cni-bincopy/0.log" Jul 24 18:33:08.385826 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:08.385809 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fzfj6_397fd802-aa1f-4a3e-bd2b-6835089bbf23/whereabouts-cni/0.log" Jul 24 18:33:08.677269 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:08.677197 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2fs78_fa9f496c-49f7-4a62-a5a9-1bb903fc553d/network-metrics-daemon/0.log" Jul 24 18:33:08.697367 ip-10-0-137-185 kubenswrapper[2561]: I0724 18:33:08.697341 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2fs78_fa9f496c-49f7-4a62-a5a9-1bb903fc553d/kube-rbac-proxy/0.log"