Jul 24 00:11:11.266596 ip-10-0-139-82 systemd[1]: Starting Kubernetes Kubelet... Jul 24 00:11:11.731862 ip-10-0-139-82 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 00:11:11.731862 ip-10-0-139-82 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 00:11:11.731862 ip-10-0-139-82 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 00:11:11.731862 ip-10-0-139-82 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 00:11:11.731862 ip-10-0-139-82 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 00:11:11.731862 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.692070 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 00:11:11.731862 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706190 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 00:11:11.731862 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706211 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 00:11:11.731862 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706214 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 00:11:11.731862 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706218 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 00:11:11.731862 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706222 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 00:11:11.731862 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706225 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 00:11:11.731862 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706229 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 00:11:11.731862 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706232 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706235 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706238 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706241 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706243 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706246 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706249 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706252 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706255 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706258 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706261 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706264 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706268 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706270 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706273 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706276 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706279 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706281 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706284 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706287 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 00:11:11.733334 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706291 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706293 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706298 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706303 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706306 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706309 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706311 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706316 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706320 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706325 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706328 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706331 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706334 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706337 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706339 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706342 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706345 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706347 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706350 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 00:11:11.733963 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706353 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706355 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706358 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706360 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706363 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706366 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706368 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706370 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706373 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706376 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706379 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706382 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706385 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706388 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706394 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706397 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706400 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706402 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706405 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706408 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 00:11:11.735259 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706410 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706413 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706415 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706418 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706421 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706423 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706427 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706432 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706435 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706437 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706440 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706443 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706446 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706449 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706451 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706454 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706457 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706459 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706462 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706465 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 00:11:11.735876 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706467 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706471 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706474 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706476 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706479 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706482 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706484 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706490 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706493 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706495 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706498 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706500 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.706503 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707356 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707363 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707366 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707368 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707372 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707375 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707377 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 00:11:11.737342 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707380 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707383 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707388 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707392 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707395 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707398 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707401 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707403 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707406 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707409 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707411 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707414 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707416 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707419 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707422 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707424 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707427 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707430 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707433 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707436 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 00:11:11.738198 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707445 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707448 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707450 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707453 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707455 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707458 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707460 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707462 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707465 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707468 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707471 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707473 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707475 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707478 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707480 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707483 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707485 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707488 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707491 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707494 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 00:11:11.739478 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707496 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707499 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707501 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707504 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707506 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707509 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707511 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707514 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707516 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707520 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707523 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707525 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707528 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707536 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707539 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707541 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707544 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707546 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707550 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 00:11:11.740266 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707554 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707557 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707560 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707563 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707565 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707568 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707570 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707573 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707575 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707580 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707582 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707585 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707589 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707592 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707595 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707598 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707600 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707603 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707605 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 00:11:11.741106 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707608 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707611 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707613 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707617 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707620 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707622 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707625 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707627 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707657 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707662 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707665 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707669 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707673 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.707677 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708106 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708111 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708114 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708117 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708120 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708123 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708126 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 24 00:11:11.741658 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708128 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708131 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708134 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708137 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708139 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708142 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708145 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708148 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708150 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708153 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708156 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708158 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708161 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708163 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708166 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708168 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708171 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708173 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708176 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708182 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 00:11:11.742226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708184 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708187 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708189 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708192 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708194 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708197 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708200 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708202 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708207 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708210 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708213 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708217 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708223 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708226 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708229 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708232 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708234 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708237 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708240 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 00:11:11.742869 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708242 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708246 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708249 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708252 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708254 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708257 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708261 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708264 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708267 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708270 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708273 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708275 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708278 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708281 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708284 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708286 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708289 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708292 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708294 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 00:11:11.743608 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708297 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708299 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708302 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708304 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708307 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708310 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708314 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708316 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708319 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708322 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708324 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708327 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708329 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708332 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708336 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708338 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708341 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708343 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708347 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708350 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 00:11:11.744260 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708353 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708363 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708367 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708370 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708373 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708377 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708379 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708383 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708385 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708388 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708390 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708393 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708396 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.708399 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708481 2560 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708489 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708496 2560 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708500 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708506 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708509 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708514 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 00:11:11.745308 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708518 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708522 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708525 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708528 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708532 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708537 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708540 2560 flags.go:64] FLAG: --cgroup-root="" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708543 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708546 2560 flags.go:64] FLAG: --client-ca-file="" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708549 2560 flags.go:64] FLAG: --cloud-provider="external" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708552 2560 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708559 2560 flags.go:64] FLAG: --cluster-domain="" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708562 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708565 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708568 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708572 2560 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708575 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708579 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708582 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708586 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708589 2560 flags.go:64] FLAG: --contention-profiling="false" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708592 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708595 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708599 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 00:11:11.745947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708601 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708606 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708608 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708611 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708614 2560 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708620 2560 flags.go:64] FLAG: --enable-server="true" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708624 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708628 2560 flags.go:64] FLAG: --event-burst="100" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708631 2560 flags.go:64] FLAG: --event-qps="50" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708634 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708652 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708655 2560 flags.go:64] FLAG: --eviction-hard="" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708659 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708664 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708667 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708670 2560 flags.go:64] FLAG: --eviction-soft="" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708673 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708676 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708679 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708682 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708685 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708688 2560 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708691 2560 flags.go:64] FLAG: --feature-gates="" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708695 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708699 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 00:11:11.747207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708702 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708705 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708709 2560 flags.go:64] FLAG: --healthz-port="10248" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708713 2560 flags.go:64] FLAG: --help="false" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708716 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-139-82.ec2.internal" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708719 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708722 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708725 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708728 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708732 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708734 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708737 2560 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708742 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708745 2560 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708748 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708751 2560 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708754 2560 flags.go:64] FLAG: --kube-reserved="" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708757 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708760 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708764 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708769 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708772 2560 flags.go:64] FLAG: --lock-file="" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708775 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708777 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 00:11:11.748029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708781 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708786 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708789 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708792 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708795 2560 flags.go:64] FLAG: --logging-format="text" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708798 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708801 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708804 2560 flags.go:64] FLAG: --manifest-url="" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708807 2560 flags.go:64] FLAG: --manifest-url-header="" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708811 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708814 2560 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708818 2560 flags.go:64] FLAG: --max-pods="110" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708822 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708826 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708829 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708832 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708835 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708838 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708841 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708849 2560 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708853 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708856 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708860 2560 flags.go:64] FLAG: --pod-cidr="" Jul 24 00:11:11.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708863 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708869 2560 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708872 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708875 2560 flags.go:64] FLAG: --pods-per-core="0" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708878 2560 flags.go:64] FLAG: --port="10250" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708881 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708886 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0359699f2384f0dfb" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708889 2560 flags.go:64] FLAG: --qos-reserved="" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708892 2560 flags.go:64] FLAG: --read-only-port="10255" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708895 2560 flags.go:64] FLAG: --register-node="true" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708898 2560 flags.go:64] FLAG: --register-with-taints="" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708901 2560 flags.go:64] FLAG: --registry-burst="10" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708904 2560 flags.go:64] FLAG: --registry-qps="5" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708907 2560 flags.go:64] FLAG: --reserved-cpus="" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708910 2560 flags.go:64] FLAG: --reserved-memory="" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708914 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708917 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708920 2560 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708923 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708926 2560 flags.go:64] FLAG: --runonce="false" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708929 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708932 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708936 2560 flags.go:64] FLAG: --seccomp-default="false" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708939 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708942 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708945 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 00:11:11.750111 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708948 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708951 2560 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708954 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708958 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708961 2560 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708964 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708967 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708970 2560 flags.go:64] FLAG: --system-cgroups="" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708973 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708979 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708981 2560 flags.go:64] FLAG: --tls-cert-file="" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708984 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708991 2560 flags.go:64] FLAG: --tls-min-version="" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708993 2560 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708996 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.708999 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.709002 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.709005 2560 flags.go:64] FLAG: --v="2" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.709009 2560 flags.go:64] FLAG: --version="false" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.709014 2560 flags.go:64] FLAG: --vmodule="" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.709018 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.709022 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709149 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709154 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 00:11:11.751017 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709157 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709160 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709163 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709166 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709169 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709172 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709175 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709177 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709180 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709182 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709185 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709190 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709193 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709196 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709199 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709202 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709205 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709208 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709211 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 00:11:11.751694 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709214 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709220 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709223 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709225 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709228 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709231 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709234 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709237 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709239 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709242 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709244 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709247 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709250 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709252 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709255 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709258 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709260 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709263 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709265 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709268 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 00:11:11.752237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709271 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709273 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709276 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709279 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709283 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709285 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709288 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709290 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709293 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709296 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709299 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709301 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709304 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709308 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709312 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709315 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709319 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709322 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709325 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 00:11:11.752892 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709327 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709331 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709333 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709335 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709338 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709340 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709343 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709346 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709349 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709351 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709354 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709357 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709359 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709362 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709365 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709367 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709370 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709373 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709376 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709378 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 00:11:11.753439 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709381 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709383 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709386 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709388 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709391 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709394 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709399 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709402 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709404 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709407 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709410 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709412 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709415 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709417 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709420 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709422 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709425 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709427 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 00:11:11.828589 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.709430 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 00:11:11.796172 ip-10-0-139-82 systemd[1]: Started Kubernetes Kubelet. Jul 24 00:11:11.830114 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.710300 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 00:11:11.830114 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.717918 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 00:11:11.830114 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.717938 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 00:11:11.830114 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.718555 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 00:11:11.830114 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.718563 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 00:11:11.830114 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718629 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 00:11:11.830114 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718652 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 00:11:11.830114 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718657 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 00:11:11.830114 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718660 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 00:11:11.830114 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718663 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 00:11:11.830114 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718667 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 00:11:11.830114 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718670 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 00:11:11.830114 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718673 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 00:11:11.830114 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718676 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 00:11:11.830114 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718679 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718681 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718684 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718687 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718689 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718692 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718694 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718697 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718700 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718702 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718705 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718708 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718710 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718713 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718719 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718723 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718725 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718728 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718731 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718734 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 00:11:11.831077 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718737 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718740 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718744 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718748 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718752 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718755 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718758 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718760 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718763 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718766 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718769 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718771 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718774 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718776 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718779 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718781 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718784 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718787 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718790 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 00:11:11.832094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718792 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718795 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718798 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718801 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718803 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718806 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718808 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718813 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718817 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718820 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718822 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718826 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718829 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718831 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718834 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718837 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718840 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718842 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718845 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718848 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 00:11:11.833290 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718851 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718853 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718856 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718858 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718861 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718863 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718867 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718870 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718873 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718876 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718879 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718882 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718885 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718888 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718891 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718894 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718896 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718899 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718901 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 00:11:11.837000 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718906 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 00:11:11.837943 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718910 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 00:11:11.837943 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718914 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 00:11:11.837943 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718918 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 24 00:11:11.837943 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718920 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 00:11:11.837943 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718923 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 00:11:11.837943 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718926 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 00:11:11.837943 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718929 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 00:11:11.837943 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718931 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 00:11:11.837943 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718934 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 00:11:11.837943 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718936 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 00:11:11.837943 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.718939 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 00:11:11.837943 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.718944 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 00:11:11.837943 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719084 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 00:11:11.837943 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719090 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719093 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719097 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719099 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719102 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719105 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719108 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719111 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719114 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719117 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719119 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719123 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719126 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719130 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719134 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719137 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719140 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719143 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719149 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 00:11:11.838643 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719151 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719155 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719158 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719161 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719164 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719167 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719170 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719174 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719177 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719180 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719183 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719185 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719188 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719192 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719195 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719198 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719200 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719203 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719206 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 00:11:11.842442 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719208 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719211 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719213 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719216 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719218 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719221 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719223 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719226 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719228 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719231 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719234 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719237 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719241 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719244 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719248 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719251 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719253 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719256 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719259 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719261 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 00:11:11.843379 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719264 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719266 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719269 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719271 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719274 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719277 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719279 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719283 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719285 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719287 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719290 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719293 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719295 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719298 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719300 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719302 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719305 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719308 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719310 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719312 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 00:11:11.844321 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719315 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719318 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719320 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719323 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719326 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719329 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719333 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719336 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719338 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719341 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719343 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719346 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719349 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719351 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719354 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719356 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719359 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719361 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719364 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 00:11:11.845519 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:11.719367 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 00:11:11.846454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.719372 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 00:11:11.846454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.719501 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 00:11:11.846454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.721796 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 00:11:11.846454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.723503 2560 server.go:1019] "Starting client certificate rotation" Jul 24 00:11:11.846454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.723610 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 00:11:11.846454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.723662 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 00:11:11.846454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.727111 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 00:11:11.846454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.730716 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 00:11:11.846454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.755045 2560 log.go:25] "Validated CRI v1 runtime API" Jul 24 00:11:11.846454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.757602 2560 log.go:25] "Validated CRI v1 image API" Jul 24 00:11:11.846454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.758404 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 00:11:11.846454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.763054 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 00:11:11.847396 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.768737 2560 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9dce503d-aefa-417f-831d-606ac3014c37:/dev/nvme0n1p4 b1b5b607-bd28-4917-b4dd-f0de64c89113:/dev/nvme0n1p3] Jul 24 00:11:11.847396 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.768754 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.773805 2560 manager.go:216] Machine: {Timestamp:2026-07-24 00:11:11.773267799 +0000 UTC m=+0.378582107 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100559 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fb3f88536f9e552c58675c21bcf02 SystemUUID:ec2fb3f8-8536-f9e5-52c5-8675c21bcf02 BootID:0e6ac2ac-9915-4fa4-927b-68a245edb19f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:bf:47:04:48:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:bf:47:04:48:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:bb:62:99:a2:ef Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.773928 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.774066 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.775180 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.775212 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-82.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.775974 2560 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.775996 2560 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.776004 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.776667 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.778108 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.781731 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.781906 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.781994 2560 kubelet.go:486] "Attempting to sync node with API server" Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.782011 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.782028 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.782044 2560 kubelet.go:398] "Adding apiserver pod source" Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.782057 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 00:11:11.848323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.782539 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.782563 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.785000 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.787517 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.787556 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.789631 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.789649 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.789657 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.789664 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.789676 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.789690 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.789700 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.789707 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.789731 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.789739 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.789757 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.789764 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.791448 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.791461 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.791976 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vlngb" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.795332 2560 server.go:1268] "Started kubelet" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.796475 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.797684 2560 server.go:310] "Adding debug handlers to kubelet server" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.812791 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vlngb" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:11.829148 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.829397 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-82.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 00:11:11.849262 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:11.829547 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-82.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 00:11:11.998558 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:11.831853 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-82.ec2.internal.18c51152230cb6b3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-82.ec2.internal,UID:ip-10-0-139-82.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-82.ec2.internal,},FirstTimestamp:2026-07-24 00:11:11.795295923 +0000 UTC m=+0.400610234,LastTimestamp:2026-07-24 00:11:11.795295923 +0000 UTC m=+0.400610234,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-82.ec2.internal,}" Jul 24 00:11:11.998558 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.810368 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 00:11:11.998558 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:11.994401 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 00:11:11.998558 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:11.996896 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.018220 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.022963 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.022970 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.023157 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-82.ec2.internal\" not found" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.023952 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.023969 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.023986 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.024058 2560 factory.go:55] Registering systemd factory Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.024074 2560 factory.go:223] Registration of the systemd container factory successfully Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.024112 2560 reconstruct.go:97] "Volume reconstruction finished" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.024124 2560 reconciler.go:29] "Reconciler: start to sync state" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.024354 2560 factory.go:153] Registering CRI-O factory Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.024370 2560 factory.go:223] Registration of the crio container factory successfully Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.024444 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.024472 2560 factory.go:103] Registering Raw factory Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.024489 2560 manager.go:1196] Started watching for new ooms in manager Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.025283 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.025296 2560 manager.go:349] Starting recovery of all containers Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.029075 2560 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-82.ec2.internal\" not found" node="ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.032489 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.035896 2560 manager.go:354] Recovery completed Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.042974 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.045141 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasSufficientMemory" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.045165 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.045176 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasSufficientPID" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.045948 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.045958 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.045977 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.048113 2560 policy_none.go:49] "None policy: Start" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.048127 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.048138 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.049311 2560 policy_none.go:47] "Start" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.050158 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.051316 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasSufficientMemory" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.051337 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 00:11:12.100404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.051347 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasSufficientPID" Jul 24 00:11:12.181825 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.125036 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-82.ec2.internal\" not found" Jul 24 00:11:12.181825 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.129565 2560 manager.go:337] "Starting Device Plugin manager" Jul 24 00:11:12.181825 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.129603 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 00:11:12.181825 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.129615 2560 server.go:87] "Starting device plugin registration server" Jul 24 00:11:12.181825 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.129963 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 00:11:12.181825 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.129975 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 00:11:12.181825 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.130075 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 00:11:12.181825 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.130191 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 00:11:12.181825 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.130200 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 00:11:12.181825 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.131819 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 00:11:12.181825 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.131885 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-82.ec2.internal\" not found" Jul 24 00:11:12.271644 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.230404 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 00:11:12.271644 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.231600 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasSufficientMemory" Jul 24 00:11:12.271644 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.231637 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 00:11:12.271644 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.231662 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasSufficientPID" Jul 24 00:11:12.271644 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.231695 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.271644 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.237522 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.271644 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.237554 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-82.ec2.internal\": node \"ip-10-0-139-82.ec2.internal\" not found" Jul 24 00:11:12.271644 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.271071 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-82.ec2.internal\" not found" Jul 24 00:11:12.341151 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.341108 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 00:11:12.341307 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.341161 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 00:11:12.341307 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.341194 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 00:11:12.341307 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.341237 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 00:11:12.343437 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.343410 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 00:11:12.372100 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.372065 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-82.ec2.internal\" not found" Jul 24 00:11:12.441721 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.441619 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-82.ec2.internal"] Jul 24 00:11:12.441793 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.441741 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 00:11:12.443730 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.443714 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasSufficientMemory" Jul 24 00:11:12.443793 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.443747 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 00:11:12.443793 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.443762 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasSufficientPID" Jul 24 00:11:12.445336 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.445322 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 00:11:12.445474 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.445441 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.445474 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.445474 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 00:11:12.446273 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.446250 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasSufficientMemory" Jul 24 00:11:12.446273 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.446281 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 00:11:12.446461 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.446291 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasSufficientPID" Jul 24 00:11:12.446461 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.446251 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasSufficientMemory" Jul 24 00:11:12.446461 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.446370 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 00:11:12.446461 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.446388 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasSufficientPID" Jul 24 00:11:12.447910 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.447893 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.448017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.447920 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 00:11:12.448786 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.448769 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasSufficientMemory" Jul 24 00:11:12.448864 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.448796 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 00:11:12.448864 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.448807 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeHasSufficientPID" Jul 24 00:11:12.465424 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.465400 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-82.ec2.internal\" not found" node="ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.469254 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.469237 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-82.ec2.internal\" not found" node="ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.472273 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.472253 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-82.ec2.internal\" not found" Jul 24 00:11:12.527733 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.527699 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71251ac2728c24e732a941d31282182a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal\" (UID: \"71251ac2728c24e732a941d31282182a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.527857 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.527738 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dd7dabc7b022df6b0fbb3ed250311918-config\") pod \"kube-apiserver-proxy-ip-10-0-139-82.ec2.internal\" (UID: \"dd7dabc7b022df6b0fbb3ed250311918\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.527857 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.527758 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71251ac2728c24e732a941d31282182a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal\" (UID: \"71251ac2728c24e732a941d31282182a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.572404 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.572370 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-82.ec2.internal\" not found" Jul 24 00:11:12.627907 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.627872 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dd7dabc7b022df6b0fbb3ed250311918-config\") pod \"kube-apiserver-proxy-ip-10-0-139-82.ec2.internal\" (UID: \"dd7dabc7b022df6b0fbb3ed250311918\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.628103 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.627909 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71251ac2728c24e732a941d31282182a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal\" (UID: \"71251ac2728c24e732a941d31282182a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.628103 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.627928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71251ac2728c24e732a941d31282182a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal\" (UID: \"71251ac2728c24e732a941d31282182a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.628103 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.627998 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71251ac2728c24e732a941d31282182a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal\" (UID: \"71251ac2728c24e732a941d31282182a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.628103 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.628009 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dd7dabc7b022df6b0fbb3ed250311918-config\") pod \"kube-apiserver-proxy-ip-10-0-139-82.ec2.internal\" (UID: \"dd7dabc7b022df6b0fbb3ed250311918\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.628103 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.627986 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71251ac2728c24e732a941d31282182a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal\" (UID: \"71251ac2728c24e732a941d31282182a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.672614 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.672572 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-82.ec2.internal\" not found" Jul 24 00:11:12.694465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.694407 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 00:11:12.723684 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.723631 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.723684 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.723681 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 00:11:12.723914 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.723892 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 00:11:12.723975 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.723893 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 00:11:12.723975 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.723893 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 00:11:12.768989 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.768948 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.773076 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.773028 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.774698 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.774677 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 00:11:12.776518 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.776503 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-82.ec2.internal" Jul 24 00:11:12.782705 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.782684 2560 apiserver.go:52] "Watching apiserver" Jul 24 00:11:12.790546 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.790522 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 00:11:12.790931 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.790907 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-lqb6d","openshift-network-diagnostics/network-check-target-2rr2t","openshift-ovn-kubernetes/ovnkube-node-rfgrr","kube-system/global-pull-secret-syncer-k66hc","kube-system/konnectivity-agent-kfmld","openshift-image-registry/node-ca-c4c45","openshift-multus/multus-additional-cni-plugins-hvwnr","openshift-network-operator/iptables-alerter-xqhjk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx","openshift-cluster-node-tuning-operator/tuned-6hw68","openshift-dns/node-resolver-wb5s8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal","openshift-multus/multus-m2mlt"] Jul 24 00:11:12.791847 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.791826 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 00:11:12.792613 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.792595 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.793970 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.793952 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:12.794054 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.794017 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:12.796949 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.796694 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.798037 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.798016 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.799531 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.799515 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.800785 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.800772 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c4c45" Jul 24 00:11:12.802271 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.802258 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:12.802334 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.802318 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:12.803971 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.803840 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xqhjk" Jul 24 00:11:12.805506 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.805482 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kfmld" Jul 24 00:11:12.805806 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.805624 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 00:11:12.805806 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.805675 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 00:11:12.806963 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.806946 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.808536 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.808518 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wb5s8" Jul 24 00:11:12.809191 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.809169 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7znpr\"" Jul 24 00:11:12.809967 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.809947 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-82.ec2.internal"] Jul 24 00:11:12.810072 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.810029 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:12.810132 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.810092 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:12.810931 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.810903 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 00:11:12.812702 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.812686 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 00:11:12.812957 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.812942 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 00:11:12.813804 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.813790 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 00:11:12.814835 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.814815 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 00:11:12.814835 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.814836 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 00:11:12.815014 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.814822 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 00:11:12.825534 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.825509 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 00:11:12.825736 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.825719 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2jzxh\"" Jul 24 00:11:12.825819 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.825764 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dzd2v\"" Jul 24 00:11:12.825876 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.825824 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7l755\"" Jul 24 00:11:12.826045 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.825985 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 00:11:12.826408 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.826386 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 00:11:12.826539 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.826430 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 00:11:12.826539 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.826454 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 00:11:12.826539 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.826486 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 00:11:12.826727 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.826549 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 00:11:12.826727 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.826618 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5rq8f\"" Jul 24 00:11:12.826727 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.826698 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zs9qf\"" Jul 24 00:11:12.826925 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.826844 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 00:11:12.826985 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.826954 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 00:11:12.826985 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.826969 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 00:11:12.827127 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.827020 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 00:11:12.827127 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.827036 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 00:11:12.827127 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.827075 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 00:11:12.827281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.827186 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 00:11:12.827281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.827258 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 00:11:12.827377 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.827279 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 00:11:12.827377 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.827261 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 00:11:12.827377 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.827335 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hcvp8\"" Jul 24 00:11:12.827499 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.827436 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2r45s\"" Jul 24 00:11:12.828817 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.828796 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-host\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.828951 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.828824 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-multus-socket-dir-parent\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.828951 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.828841 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/567139c0-866a-4ff0-9085-579046198698-host\") pod \"node-ca-c4c45\" (UID: \"567139c0-866a-4ff0-9085-579046198698\") " pod="openshift-image-registry/node-ca-c4c45" Jul 24 00:11:12.828951 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.828858 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glzks\" (UniqueName: \"kubernetes.io/projected/3266eb6f-f15e-4b80-ba25-b0f9807214e6-kube-api-access-glzks\") pod \"network-metrics-daemon-lqb6d\" (UID: \"3266eb6f-f15e-4b80-ba25-b0f9807214e6\") " pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:12.828951 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.828918 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ff0d6407-0175-4c0e-a54c-528dae80387a-kubelet-config\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:12.829170 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.828959 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.829170 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.828991 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-registration-dir\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.829170 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829018 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-var-lib-kubelet\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.829170 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829044 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz5qx\" (UniqueName: \"kubernetes.io/projected/354dff89-24c0-40e7-80ef-7d3c86257772-kube-api-access-wz5qx\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.829170 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829067 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/08ddb057-b851-403c-84e8-f2c50818ceaf-env-overrides\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.829170 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829086 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-etc-kubernetes\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.829170 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829153 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-var-lib-openvswitch\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.829634 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829211 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/08ddb057-b851-403c-84e8-f2c50818ceaf-ovn-node-metrics-cert\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.829634 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829233 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff1028bf-22cb-4b54-9539-ab4ed63382fc-host-slash\") pod \"iptables-alerter-xqhjk\" (UID: \"ff1028bf-22cb-4b54-9539-ab4ed63382fc\") " pod="openshift-network-operator/iptables-alerter-xqhjk" Jul 24 00:11:12.829634 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829265 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-sysconfig\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.829634 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829290 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-sysctl-d\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.829634 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829339 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15937a1b-bb3c-46a5-8b30-3d2e4c260be3-tmp-dir\") pod \"node-resolver-wb5s8\" (UID: \"15937a1b-bb3c-46a5-8b30-3d2e4c260be3\") " pod="openshift-dns/node-resolver-wb5s8" Jul 24 00:11:12.829634 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829379 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:12.829634 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829408 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6bdd868-f7ad-4866-965a-d60339268678-cni-binary-copy\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.829634 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829435 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-systemd-units\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.829634 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829460 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-slash\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.829634 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829485 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-run-systemd\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.829634 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829508 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6b5ccef-2d64-4b60-833b-987d428e4856-tmp\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.829634 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.829535 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-run-netns\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.831943 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.831912 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/08ddb057-b851-403c-84e8-f2c50818ceaf-ovnkube-config\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.832118 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.831967 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n462z\" (UniqueName: \"kubernetes.io/projected/08ddb057-b851-403c-84e8-f2c50818ceaf-kube-api-access-n462z\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.832118 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832003 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx9s7\" (UniqueName: \"kubernetes.io/projected/ff1028bf-22cb-4b54-9539-ab4ed63382fc-kube-api-access-lx9s7\") pod \"iptables-alerter-xqhjk\" (UID: \"ff1028bf-22cb-4b54-9539-ab4ed63382fc\") " pod="openshift-network-operator/iptables-alerter-xqhjk" Jul 24 00:11:12.832118 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832035 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-sysctl-conf\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.832118 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832066 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-os-release\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.832118 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832096 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-run-k8s-cni-cncf-io\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.832465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832121 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/354dff89-24c0-40e7-80ef-7d3c86257772-multus-daemon-config\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.832465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832153 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4krhf\" (UniqueName: \"kubernetes.io/projected/567139c0-866a-4ff0-9085-579046198698-kube-api-access-4krhf\") pod \"node-ca-c4c45\" (UID: \"567139c0-866a-4ff0-9085-579046198698\") " pod="openshift-image-registry/node-ca-c4c45" Jul 24 00:11:12.832465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832183 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-systemd\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.832465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832215 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ff0d6407-0175-4c0e-a54c-528dae80387a-dbus\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:12.832465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832246 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-var-lib-cni-bin\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.832465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832273 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/567139c0-866a-4ff0-9085-579046198698-serviceca\") pod \"node-ca-c4c45\" (UID: \"567139c0-866a-4ff0-9085-579046198698\") " pod="openshift-image-registry/node-ca-c4c45" Jul 24 00:11:12.832465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832302 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-kubelet\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.832465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832332 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-run-netns\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.832465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832361 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-cni-netd\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.832465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832391 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/08ddb057-b851-403c-84e8-f2c50818ceaf-ovnkube-script-lib\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.832465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832424 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f75ct\" (UniqueName: \"kubernetes.io/projected/15937a1b-bb3c-46a5-8b30-3d2e4c260be3-kube-api-access-f75ct\") pod \"node-resolver-wb5s8\" (UID: \"15937a1b-bb3c-46a5-8b30-3d2e4c260be3\") " pod="openshift-dns/node-resolver-wb5s8" Jul 24 00:11:12.832465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832454 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-sys-fs\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.833229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832485 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-system-cni-dir\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.833229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832517 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e6bdd868-f7ad-4866-965a-d60339268678-os-release\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.833229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832623 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6bdd868-f7ad-4866-965a-d60339268678-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.833229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff1028bf-22cb-4b54-9539-ab4ed63382fc-iptables-alerter-script\") pod \"iptables-alerter-xqhjk\" (UID: \"ff1028bf-22cb-4b54-9539-ab4ed63382fc\") " pod="openshift-network-operator/iptables-alerter-xqhjk" Jul 24 00:11:12.833229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832769 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mjjr\" (UniqueName: \"kubernetes.io/projected/e6bdd868-f7ad-4866-965a-d60339268678-kube-api-access-9mjjr\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.833229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832830 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vxsd9\"" Jul 24 00:11:12.833229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832845 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-etc-openvswitch\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.833229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.832890 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.833229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833030 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 00:11:12.833229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833027 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-modprobe-d\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.833229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833100 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-etc-selinux\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.833229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833103 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 00:11:12.833229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833147 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-node-log\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.833760 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833270 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-cnibin\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.833760 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833349 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-multus-conf-dir\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.833760 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833406 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e6bdd868-f7ad-4866-965a-d60339268678-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.833760 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833459 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-log-socket\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.833760 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833509 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-kubernetes\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.833760 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833529 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-lib-modules\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.833760 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833556 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/87ccd406-4359-435e-8c88-9e677b0da0ec-agent-certs\") pod \"konnectivity-agent-kfmld\" (UID: \"87ccd406-4359-435e-8c88-9e677b0da0ec\") " pod="kube-system/konnectivity-agent-kfmld" Jul 24 00:11:12.833760 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833574 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cthx\" (UniqueName: \"kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx\") pod \"network-check-target-2rr2t\" (UID: \"0b1308db-61d6-49a8-b546-cf5bff91afb5\") " pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:12.833760 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833593 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-sys\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.833760 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833612 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-var-lib-kubelet\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.833760 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833630 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/15937a1b-bb3c-46a5-8b30-3d2e4c260be3-hosts-file\") pod \"node-resolver-wb5s8\" (UID: \"15937a1b-bb3c-46a5-8b30-3d2e4c260be3\") " pod="openshift-dns/node-resolver-wb5s8" Jul 24 00:11:12.834099 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.833653 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-socket-dir\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.834099 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834035 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-hostroot\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.834099 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834080 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-tuned\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.834245 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834130 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbwx7\" (UniqueName: \"kubernetes.io/projected/e6b5ccef-2d64-4b60-833b-987d428e4856-kube-api-access-zbwx7\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.834245 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834166 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-multus-cni-dir\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.834245 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834195 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/354dff89-24c0-40e7-80ef-7d3c86257772-cni-binary-copy\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.834245 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834226 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6bdd868-f7ad-4866-965a-d60339268678-system-cni-dir\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.834419 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834248 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6bdd868-f7ad-4866-965a-d60339268678-cnibin\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.834419 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834264 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6bdd868-f7ad-4866-965a-d60339268678-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.834419 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834280 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-run-openvswitch\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.834419 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834322 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-device-dir\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.834419 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834358 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-var-lib-cni-multus\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.834419 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834398 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-run-multus-certs\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.834653 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834429 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs\") pod \"network-metrics-daemon-lqb6d\" (UID: \"3266eb6f-f15e-4b80-ba25-b0f9807214e6\") " pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:12.834653 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834457 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-run\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.834653 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834480 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jq9d\" (UniqueName: \"kubernetes.io/projected/95e1aec3-8f32-47c9-9854-c63562733dbe-kube-api-access-4jq9d\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.834653 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834496 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/87ccd406-4359-435e-8c88-9e677b0da0ec-konnectivity-ca\") pod \"konnectivity-agent-kfmld\" (UID: \"87ccd406-4359-435e-8c88-9e677b0da0ec\") " pod="kube-system/konnectivity-agent-kfmld" Jul 24 00:11:12.834653 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834513 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-run-ovn\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.834653 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834534 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-run-ovn-kubernetes\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.834653 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.834573 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-cni-bin\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.842151 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.842132 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 00:11:12.935783 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.935736 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-systemd\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.935783 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.935785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ff0d6407-0175-4c0e-a54c-528dae80387a-dbus\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:12.936000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.935805 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-var-lib-cni-bin\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.936000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.935822 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/567139c0-866a-4ff0-9085-579046198698-serviceca\") pod \"node-ca-c4c45\" (UID: \"567139c0-866a-4ff0-9085-579046198698\") " pod="openshift-image-registry/node-ca-c4c45" Jul 24 00:11:12.936000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.935836 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-kubelet\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.936000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.935875 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-kubelet\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.936000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.935887 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-var-lib-cni-bin\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.936000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.935904 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-run-netns\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.936000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.935922 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-cni-netd\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.936000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.935885 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-systemd\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.936000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.935946 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-run-netns\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.936000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.935937 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/08ddb057-b851-403c-84e8-f2c50818ceaf-ovnkube-script-lib\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.936000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.935968 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-cni-netd\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.936000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.935996 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f75ct\" (UniqueName: \"kubernetes.io/projected/15937a1b-bb3c-46a5-8b30-3d2e4c260be3-kube-api-access-f75ct\") pod \"node-resolver-wb5s8\" (UID: \"15937a1b-bb3c-46a5-8b30-3d2e4c260be3\") " pod="openshift-dns/node-resolver-wb5s8" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936020 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ff0d6407-0175-4c0e-a54c-528dae80387a-dbus\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936027 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-sys-fs\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936054 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-system-cni-dir\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936087 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-sys-fs\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936120 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-system-cni-dir\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936192 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e6bdd868-f7ad-4866-965a-d60339268678-os-release\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936216 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6bdd868-f7ad-4866-965a-d60339268678-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936235 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff1028bf-22cb-4b54-9539-ab4ed63382fc-iptables-alerter-script\") pod \"iptables-alerter-xqhjk\" (UID: \"ff1028bf-22cb-4b54-9539-ab4ed63382fc\") " pod="openshift-network-operator/iptables-alerter-xqhjk" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936264 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mjjr\" (UniqueName: \"kubernetes.io/projected/e6bdd868-f7ad-4866-965a-d60339268678-kube-api-access-9mjjr\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936273 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e6bdd868-f7ad-4866-965a-d60339268678-os-release\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936289 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-etc-openvswitch\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936319 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936345 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-modprobe-d\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936368 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-etc-selinux\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936381 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/567139c0-866a-4ff0-9085-579046198698-serviceca\") pod \"node-ca-c4c45\" (UID: \"567139c0-866a-4ff0-9085-579046198698\") " pod="openshift-image-registry/node-ca-c4c45" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936418 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-etc-openvswitch\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.936582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936453 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-node-log\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936475 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-etc-selinux\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936483 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-cnibin\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936490 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/08ddb057-b851-403c-84e8-f2c50818ceaf-ovnkube-script-lib\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936513 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-multus-conf-dir\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936544 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e6bdd868-f7ad-4866-965a-d60339268678-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936549 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-cnibin\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936575 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-modprobe-d\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936572 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-log-socket\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936581 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-node-log\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936617 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-log-socket\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936647 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-multus-conf-dir\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936707 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936736 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-kubernetes\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936772 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-lib-modules\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936801 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/87ccd406-4359-435e-8c88-9e677b0da0ec-agent-certs\") pod \"konnectivity-agent-kfmld\" (UID: \"87ccd406-4359-435e-8c88-9e677b0da0ec\") " pod="kube-system/konnectivity-agent-kfmld" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936807 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-kubernetes\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936828 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cthx\" (UniqueName: \"kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx\") pod \"network-check-target-2rr2t\" (UID: \"0b1308db-61d6-49a8-b546-cf5bff91afb5\") " pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:12.937497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936854 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-sys\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936879 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-var-lib-kubelet\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936902 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/15937a1b-bb3c-46a5-8b30-3d2e4c260be3-hosts-file\") pod \"node-resolver-wb5s8\" (UID: \"15937a1b-bb3c-46a5-8b30-3d2e4c260be3\") " pod="openshift-dns/node-resolver-wb5s8" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936906 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6bdd868-f7ad-4866-965a-d60339268678-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936922 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff1028bf-22cb-4b54-9539-ab4ed63382fc-iptables-alerter-script\") pod \"iptables-alerter-xqhjk\" (UID: \"ff1028bf-22cb-4b54-9539-ab4ed63382fc\") " pod="openshift-network-operator/iptables-alerter-xqhjk" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-socket-dir\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936928 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-lib-modules\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936948 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-sys\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936973 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-var-lib-kubelet\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.936980 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/15937a1b-bb3c-46a5-8b30-3d2e4c260be3-hosts-file\") pod \"node-resolver-wb5s8\" (UID: \"15937a1b-bb3c-46a5-8b30-3d2e4c260be3\") " pod="openshift-dns/node-resolver-wb5s8" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937005 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-hostroot\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937044 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-socket-dir\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937054 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-tuned\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937062 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e6bdd868-f7ad-4866-965a-d60339268678-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937025 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-hostroot\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937084 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbwx7\" (UniqueName: \"kubernetes.io/projected/e6b5ccef-2d64-4b60-833b-987d428e4856-kube-api-access-zbwx7\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937127 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-multus-cni-dir\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937152 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/354dff89-24c0-40e7-80ef-7d3c86257772-cni-binary-copy\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.938241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937190 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937225 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-multus-cni-dir\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6bdd868-f7ad-4866-965a-d60339268678-system-cni-dir\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6bdd868-f7ad-4866-965a-d60339268678-cnibin\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937298 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6bdd868-f7ad-4866-965a-d60339268678-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937322 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-run-openvswitch\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937350 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-device-dir\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937359 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6bdd868-f7ad-4866-965a-d60339268678-cnibin\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937373 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6bdd868-f7ad-4866-965a-d60339268678-system-cni-dir\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937403 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-var-lib-cni-multus\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937412 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-run-openvswitch\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937429 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-run-multus-certs\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937439 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-device-dir\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937453 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-var-lib-cni-multus\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937458 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs\") pod \"network-metrics-daemon-lqb6d\" (UID: \"3266eb6f-f15e-4b80-ba25-b0f9807214e6\") " pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937491 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-run\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937505 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-run-multus-certs\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.939176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937507 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6bdd868-f7ad-4866-965a-d60339268678-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937519 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jq9d\" (UniqueName: \"kubernetes.io/projected/95e1aec3-8f32-47c9-9854-c63562733dbe-kube-api-access-4jq9d\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.937554 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937560 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-run\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937584 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/87ccd406-4359-435e-8c88-9e677b0da0ec-konnectivity-ca\") pod \"konnectivity-agent-kfmld\" (UID: \"87ccd406-4359-435e-8c88-9e677b0da0ec\") " pod="kube-system/konnectivity-agent-kfmld" Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.937645 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs podName:3266eb6f-f15e-4b80-ba25-b0f9807214e6 nodeName:}" failed. No retries permitted until 2026-07-24 00:11:13.437607198 +0000 UTC m=+2.042921494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs") pod "network-metrics-daemon-lqb6d" (UID: "3266eb6f-f15e-4b80-ba25-b0f9807214e6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937662 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-run-ovn\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937705 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-run-ovn-kubernetes\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937729 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-cni-bin\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937737 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/354dff89-24c0-40e7-80ef-7d3c86257772-cni-binary-copy\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937752 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-host\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937776 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-multus-socket-dir-parent\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937790 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-run-ovn\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937797 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-run-ovn-kubernetes\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937796 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/567139c0-866a-4ff0-9085-579046198698-host\") pod \"node-ca-c4c45\" (UID: \"567139c0-866a-4ff0-9085-579046198698\") " pod="openshift-image-registry/node-ca-c4c45" Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937840 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-cni-bin\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937863 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-host\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.940006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937868 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-multus-socket-dir-parent\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937860 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glzks\" (UniqueName: \"kubernetes.io/projected/3266eb6f-f15e-4b80-ba25-b0f9807214e6-kube-api-access-glzks\") pod \"network-metrics-daemon-lqb6d\" (UID: \"3266eb6f-f15e-4b80-ba25-b0f9807214e6\") " pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937910 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ff0d6407-0175-4c0e-a54c-528dae80387a-kubelet-config\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937878 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/567139c0-866a-4ff0-9085-579046198698-host\") pod \"node-ca-c4c45\" (UID: \"567139c0-866a-4ff0-9085-579046198698\") " pod="openshift-image-registry/node-ca-c4c45" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.937967 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ff0d6407-0175-4c0e-a54c-528dae80387a-kubelet-config\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938034 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938066 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-registration-dir\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938094 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-var-lib-kubelet\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938114 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938123 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz5qx\" (UniqueName: \"kubernetes.io/projected/354dff89-24c0-40e7-80ef-7d3c86257772-kube-api-access-wz5qx\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938147 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/95e1aec3-8f32-47c9-9854-c63562733dbe-registration-dir\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938156 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/87ccd406-4359-435e-8c88-9e677b0da0ec-konnectivity-ca\") pod \"konnectivity-agent-kfmld\" (UID: \"87ccd406-4359-435e-8c88-9e677b0da0ec\") " pod="kube-system/konnectivity-agent-kfmld" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938157 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/08ddb057-b851-403c-84e8-f2c50818ceaf-env-overrides\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938194 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-etc-kubernetes\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938213 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-var-lib-openvswitch\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938230 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/08ddb057-b851-403c-84e8-f2c50818ceaf-ovn-node-metrics-cert\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938247 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff1028bf-22cb-4b54-9539-ab4ed63382fc-host-slash\") pod \"iptables-alerter-xqhjk\" (UID: \"ff1028bf-22cb-4b54-9539-ab4ed63382fc\") " pod="openshift-network-operator/iptables-alerter-xqhjk" Jul 24 00:11:12.940869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938266 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-sysconfig\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938292 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-sysctl-d\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938302 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-var-lib-openvswitch\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938313 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-var-lib-kubelet\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938342 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-etc-kubernetes\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938390 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-sysconfig\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938429 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff1028bf-22cb-4b54-9539-ab4ed63382fc-host-slash\") pod \"iptables-alerter-xqhjk\" (UID: \"ff1028bf-22cb-4b54-9539-ab4ed63382fc\") " pod="openshift-network-operator/iptables-alerter-xqhjk" Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938316 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15937a1b-bb3c-46a5-8b30-3d2e4c260be3-tmp-dir\") pod \"node-resolver-wb5s8\" (UID: \"15937a1b-bb3c-46a5-8b30-3d2e4c260be3\") " pod="openshift-dns/node-resolver-wb5s8" Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938465 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938470 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-sysctl-d\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938513 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6bdd868-f7ad-4866-965a-d60339268678-cni-binary-copy\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938530 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/08ddb057-b851-403c-84e8-f2c50818ceaf-env-overrides\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938556 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-systemd-units\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.938582 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938587 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-slash\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938614 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15937a1b-bb3c-46a5-8b30-3d2e4c260be3-tmp-dir\") pod \"node-resolver-wb5s8\" (UID: \"15937a1b-bb3c-46a5-8b30-3d2e4c260be3\") " pod="openshift-dns/node-resolver-wb5s8" Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.938632 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret podName:ff0d6407-0175-4c0e-a54c-528dae80387a nodeName:}" failed. No retries permitted until 2026-07-24 00:11:13.438615787 +0000 UTC m=+2.043930096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret") pod "global-pull-secret-syncer-k66hc" (UID: "ff0d6407-0175-4c0e-a54c-528dae80387a") : object "kube-system"/"original-pull-secret" not registered Jul 24 00:11:12.941463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938615 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-run-systemd\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938685 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6b5ccef-2d64-4b60-833b-987d428e4856-tmp\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938699 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-systemd-units\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938717 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-host-slash\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938762 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-run-netns\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938747 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/08ddb057-b851-403c-84e8-f2c50818ceaf-run-systemd\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938803 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/08ddb057-b851-403c-84e8-f2c50818ceaf-ovnkube-config\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938848 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-run-netns\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938882 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n462z\" (UniqueName: \"kubernetes.io/projected/08ddb057-b851-403c-84e8-f2c50818ceaf-kube-api-access-n462z\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938914 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx9s7\" (UniqueName: \"kubernetes.io/projected/ff1028bf-22cb-4b54-9539-ab4ed63382fc-kube-api-access-lx9s7\") pod \"iptables-alerter-xqhjk\" (UID: \"ff1028bf-22cb-4b54-9539-ab4ed63382fc\") " pod="openshift-network-operator/iptables-alerter-xqhjk" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938942 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-sysctl-conf\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.938975 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-os-release\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.939004 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-run-k8s-cni-cncf-io\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.939032 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/354dff89-24c0-40e7-80ef-7d3c86257772-multus-daemon-config\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.939062 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4krhf\" (UniqueName: \"kubernetes.io/projected/567139c0-866a-4ff0-9085-579046198698-kube-api-access-4krhf\") pod \"node-ca-c4c45\" (UID: \"567139c0-866a-4ff0-9085-579046198698\") " pod="openshift-image-registry/node-ca-c4c45" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.939082 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6bdd868-f7ad-4866-965a-d60339268678-cni-binary-copy\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.939226 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-host-run-k8s-cni-cncf-io\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.939253 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-sysctl-conf\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.942001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.939300 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/354dff89-24c0-40e7-80ef-7d3c86257772-os-release\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.942552 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.939308 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/08ddb057-b851-403c-84e8-f2c50818ceaf-ovnkube-config\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.942552 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.939784 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/354dff89-24c0-40e7-80ef-7d3c86257772-multus-daemon-config\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.942552 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.941228 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e6b5ccef-2d64-4b60-833b-987d428e4856-etc-tuned\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.942552 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.941277 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6b5ccef-2d64-4b60-833b-987d428e4856-tmp\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.942552 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.941347 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/08ddb057-b851-403c-84e8-f2c50818ceaf-ovn-node-metrics-cert\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.942552 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.941479 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/87ccd406-4359-435e-8c88-9e677b0da0ec-agent-certs\") pod \"konnectivity-agent-kfmld\" (UID: \"87ccd406-4359-435e-8c88-9e677b0da0ec\") " pod="kube-system/konnectivity-agent-kfmld" Jul 24 00:11:12.954601 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.954521 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz5qx\" (UniqueName: \"kubernetes.io/projected/354dff89-24c0-40e7-80ef-7d3c86257772-kube-api-access-wz5qx\") pod \"multus-m2mlt\" (UID: \"354dff89-24c0-40e7-80ef-7d3c86257772\") " pod="openshift-multus/multus-m2mlt" Jul 24 00:11:12.955865 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.955845 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f75ct\" (UniqueName: \"kubernetes.io/projected/15937a1b-bb3c-46a5-8b30-3d2e4c260be3-kube-api-access-f75ct\") pod \"node-resolver-wb5s8\" (UID: \"15937a1b-bb3c-46a5-8b30-3d2e4c260be3\") " pod="openshift-dns/node-resolver-wb5s8" Jul 24 00:11:12.958581 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.958558 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 00:11:12.958581 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.958583 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 00:11:12.958827 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.958599 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4cthx for pod openshift-network-diagnostics/network-check-target-2rr2t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 00:11:12.958827 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:12.958701 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx podName:0b1308db-61d6-49a8-b546-cf5bff91afb5 nodeName:}" failed. No retries permitted until 2026-07-24 00:11:13.458678717 +0000 UTC m=+2.063993040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4cthx" (UniqueName: "kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx") pod "network-check-target-2rr2t" (UID: "0b1308db-61d6-49a8-b546-cf5bff91afb5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 00:11:12.960366 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.960340 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4krhf\" (UniqueName: \"kubernetes.io/projected/567139c0-866a-4ff0-9085-579046198698-kube-api-access-4krhf\") pod \"node-ca-c4c45\" (UID: \"567139c0-866a-4ff0-9085-579046198698\") " pod="openshift-image-registry/node-ca-c4c45" Jul 24 00:11:12.960893 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.960873 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jq9d\" (UniqueName: \"kubernetes.io/projected/95e1aec3-8f32-47c9-9854-c63562733dbe-kube-api-access-4jq9d\") pod \"aws-ebs-csi-driver-node-9c4hx\" (UID: \"95e1aec3-8f32-47c9-9854-c63562733dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:12.961605 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.961588 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx9s7\" (UniqueName: \"kubernetes.io/projected/ff1028bf-22cb-4b54-9539-ab4ed63382fc-kube-api-access-lx9s7\") pod \"iptables-alerter-xqhjk\" (UID: \"ff1028bf-22cb-4b54-9539-ab4ed63382fc\") " pod="openshift-network-operator/iptables-alerter-xqhjk" Jul 24 00:11:12.962022 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.962001 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbwx7\" (UniqueName: \"kubernetes.io/projected/e6b5ccef-2d64-4b60-833b-987d428e4856-kube-api-access-zbwx7\") pod \"tuned-6hw68\" (UID: \"e6b5ccef-2d64-4b60-833b-987d428e4856\") " pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:12.964557 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.964528 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glzks\" (UniqueName: \"kubernetes.io/projected/3266eb6f-f15e-4b80-ba25-b0f9807214e6-kube-api-access-glzks\") pod \"network-metrics-daemon-lqb6d\" (UID: \"3266eb6f-f15e-4b80-ba25-b0f9807214e6\") " pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:12.964790 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.964769 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mjjr\" (UniqueName: \"kubernetes.io/projected/e6bdd868-f7ad-4866-965a-d60339268678-kube-api-access-9mjjr\") pod \"multus-additional-cni-plugins-hvwnr\" (UID: \"e6bdd868-f7ad-4866-965a-d60339268678\") " pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:12.964965 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.964950 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n462z\" (UniqueName: \"kubernetes.io/projected/08ddb057-b851-403c-84e8-f2c50818ceaf-kube-api-access-n462z\") pod \"ovnkube-node-rfgrr\" (UID: \"08ddb057-b851-403c-84e8-f2c50818ceaf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:12.995910 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.995836 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 00:06:12 +0000 UTC" deadline="2028-02-13 17:22:45.188655286 +0000 UTC" Jul 24 00:11:12.995910 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:12.995895 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13673h11m32.192762868s" Jul 24 00:11:13.023288 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.023250 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 00:11:13.054447 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.054413 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 00:11:13.076866 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.076830 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8vvqh" Jul 24 00:11:13.086067 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.086038 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8vvqh" Jul 24 00:11:13.127782 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.127748 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" Jul 24 00:11:13.139971 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.139947 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:13.158365 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.158335 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m2mlt" Jul 24 00:11:13.161634 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.161610 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hvwnr" Jul 24 00:11:13.166594 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.166576 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c4c45" Jul 24 00:11:13.189603 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.189566 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xqhjk" Jul 24 00:11:13.204927 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.204846 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kfmld" Jul 24 00:11:13.211009 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.210980 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6hw68" Jul 24 00:11:13.214945 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.214924 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wb5s8" Jul 24 00:11:13.443917 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.443876 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs\") pod \"network-metrics-daemon-lqb6d\" (UID: \"3266eb6f-f15e-4b80-ba25-b0f9807214e6\") " pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:13.443917 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.443926 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:13.444242 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:13.444033 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 00:11:13.444242 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:13.444088 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret podName:ff0d6407-0175-4c0e-a54c-528dae80387a nodeName:}" failed. No retries permitted until 2026-07-24 00:11:14.444072433 +0000 UTC m=+3.049386729 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret") pod "global-pull-secret-syncer-k66hc" (UID: "ff0d6407-0175-4c0e-a54c-528dae80387a") : object "kube-system"/"original-pull-secret" not registered Jul 24 00:11:13.444242 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:13.444166 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 00:11:13.444370 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:13.444244 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs podName:3266eb6f-f15e-4b80-ba25-b0f9807214e6 nodeName:}" failed. No retries permitted until 2026-07-24 00:11:14.44422774 +0000 UTC m=+3.049542036 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs") pod "network-metrics-daemon-lqb6d" (UID: "3266eb6f-f15e-4b80-ba25-b0f9807214e6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 00:11:13.545026 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.544989 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cthx\" (UniqueName: \"kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx\") pod \"network-check-target-2rr2t\" (UID: \"0b1308db-61d6-49a8-b546-cf5bff91afb5\") " pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:13.545194 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:13.545133 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 00:11:13.545194 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:13.545151 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 00:11:13.545194 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:13.545161 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4cthx for pod openshift-network-diagnostics/network-check-target-2rr2t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 00:11:13.545302 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:13.545220 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx podName:0b1308db-61d6-49a8-b546-cf5bff91afb5 nodeName:}" failed. No retries permitted until 2026-07-24 00:11:14.545202072 +0000 UTC m=+3.150516375 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4cthx" (UniqueName: "kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx") pod "network-check-target-2rr2t" (UID: "0b1308db-61d6-49a8-b546-cf5bff91afb5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 00:11:13.573253 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:13.573202 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71251ac2728c24e732a941d31282182a.slice/crio-645bf747a6230d0e4a60519d595189b2a683433ed453959bc125b22bd117034e WatchSource:0}: Error finding container 645bf747a6230d0e4a60519d595189b2a683433ed453959bc125b22bd117034e: Status 404 returned error can't find the container with id 645bf747a6230d0e4a60519d595189b2a683433ed453959bc125b22bd117034e Jul 24 00:11:13.575771 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:13.575736 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod354dff89_24c0_40e7_80ef_7d3c86257772.slice/crio-c3917a1c91c29c491bee208e6786cf0fd61c231ae810fda0f302569f9928ace2 WatchSource:0}: Error finding container c3917a1c91c29c491bee208e6786cf0fd61c231ae810fda0f302569f9928ace2: Status 404 returned error can't find the container with id c3917a1c91c29c491bee208e6786cf0fd61c231ae810fda0f302569f9928ace2 Jul 24 00:11:13.576312 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:13.576276 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod567139c0_866a_4ff0_9085_579046198698.slice/crio-33a4dd61152e002d06a7d3437a4a4404ff164beb24ee5327c105984b09874757 WatchSource:0}: Error finding container 33a4dd61152e002d06a7d3437a4a4404ff164beb24ee5327c105984b09874757: Status 404 returned error can't find the container with id 33a4dd61152e002d06a7d3437a4a4404ff164beb24ee5327c105984b09874757 Jul 24 00:11:13.584587 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.584427 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 00:11:13.587581 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:13.587560 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff1028bf_22cb_4b54_9539_ab4ed63382fc.slice/crio-2fe31821ad52fc0a3935a92dcfcb83a06a95a3d61b2d700151822c0a4a5f3547 WatchSource:0}: Error finding container 2fe31821ad52fc0a3935a92dcfcb83a06a95a3d61b2d700151822c0a4a5f3547: Status 404 returned error can't find the container with id 2fe31821ad52fc0a3935a92dcfcb83a06a95a3d61b2d700151822c0a4a5f3547 Jul 24 00:11:13.604423 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:13.604392 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd7dabc7b022df6b0fbb3ed250311918.slice/crio-2f1275f5f0ff1d39955704fa1ed6f0a1f726422f407e88be4f5d095ba2251fb2 WatchSource:0}: Error finding container 2f1275f5f0ff1d39955704fa1ed6f0a1f726422f407e88be4f5d095ba2251fb2: Status 404 returned error can't find the container with id 2f1275f5f0ff1d39955704fa1ed6f0a1f726422f407e88be4f5d095ba2251fb2 Jul 24 00:11:13.638099 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:13.638070 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b5ccef_2d64_4b60_833b_987d428e4856.slice/crio-afe74adca257bf381384b9b35124df97fade59d17f45ba886d5ae590a54388fc WatchSource:0}: Error finding container afe74adca257bf381384b9b35124df97fade59d17f45ba886d5ae590a54388fc: Status 404 returned error can't find the container with id afe74adca257bf381384b9b35124df97fade59d17f45ba886d5ae590a54388fc Jul 24 00:11:13.646511 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:13.646486 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08ddb057_b851_403c_84e8_f2c50818ceaf.slice/crio-758d8ede9e7a5e483402f0ee08991636919467fab35d97f751d135e652c25997 WatchSource:0}: Error finding container 758d8ede9e7a5e483402f0ee08991636919467fab35d97f751d135e652c25997: Status 404 returned error can't find the container with id 758d8ede9e7a5e483402f0ee08991636919467fab35d97f751d135e652c25997 Jul 24 00:11:13.647053 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:13.646998 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15937a1b_bb3c_46a5_8b30_3d2e4c260be3.slice/crio-bc27ad98b570e14246b066a3f514adbd7a8254de96551d2760bea5a4e68bc142 WatchSource:0}: Error finding container bc27ad98b570e14246b066a3f514adbd7a8254de96551d2760bea5a4e68bc142: Status 404 returned error can't find the container with id bc27ad98b570e14246b066a3f514adbd7a8254de96551d2760bea5a4e68bc142 Jul 24 00:11:13.681372 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:13.681332 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87ccd406_4359_435e_8c88_9e677b0da0ec.slice/crio-591490dae06b36641346ffe719e081493ac6c9a1c6ba1bb87bdffe1df0a363f1 WatchSource:0}: Error finding container 591490dae06b36641346ffe719e081493ac6c9a1c6ba1bb87bdffe1df0a363f1: Status 404 returned error can't find the container with id 591490dae06b36641346ffe719e081493ac6c9a1c6ba1bb87bdffe1df0a363f1 Jul 24 00:11:13.701849 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:13.701819 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6bdd868_f7ad_4866_965a_d60339268678.slice/crio-ff730db1e6cf0c7bd0b1ee26b6c87bf63c708c0e3ff09b9ce1726d377e4d1edf WatchSource:0}: Error finding container ff730db1e6cf0c7bd0b1ee26b6c87bf63c708c0e3ff09b9ce1726d377e4d1edf: Status 404 returned error can't find the container with id ff730db1e6cf0c7bd0b1ee26b6c87bf63c708c0e3ff09b9ce1726d377e4d1edf Jul 24 00:11:13.727705 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:13.727677 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 00:11:13.736109 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:13.736083 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95e1aec3_8f32_47c9_9854_c63562733dbe.slice/crio-e74a8c5718a38259daee126c4164c4213fd87004c0fc2742a79197ecf3b53779 WatchSource:0}: Error finding container e74a8c5718a38259daee126c4164c4213fd87004c0fc2742a79197ecf3b53779: Status 404 returned error can't find the container with id e74a8c5718a38259daee126c4164c4213fd87004c0fc2742a79197ecf3b53779 Jul 24 00:11:13.737964 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:13.737941 2560 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-driver start failed in pod aws-ebs-csi-driver-node-9c4hx_openshift-cluster-csi-drivers(95e1aec3-8f32-47c9-9854-c63562733dbe): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 24 00:11:13.738661 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:13.738644 2560 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-node-driver-registrar start failed in pod aws-ebs-csi-driver-node-9c4hx_openshift-cluster-csi-drivers(95e1aec3-8f32-47c9-9854-c63562733dbe): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 24 00:11:13.740433 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:13.740416 2560 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-liveness-probe start failed in pod aws-ebs-csi-driver-node-9c4hx_openshift-cluster-csi-drivers(95e1aec3-8f32-47c9-9854-c63562733dbe): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 24 00:11:13.740484 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:13.740455 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" podUID="95e1aec3-8f32-47c9-9854-c63562733dbe" Jul 24 00:11:14.091711 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.091664 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 00:06:13 +0000 UTC" deadline="2028-03-04 11:59:00.595861278 +0000 UTC" Jul 24 00:11:14.091711 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.091702 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14147h47m46.504163466s" Jul 24 00:11:14.193614 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.193530 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 00:11:14.344948 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.344568 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:14.344948 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:14.344752 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:14.345250 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.345216 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:14.345405 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:14.345320 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:14.345493 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.345465 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:14.345581 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:14.345562 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:14.355167 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.355108 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c4c45" event={"ID":"567139c0-866a-4ff0-9085-579046198698","Type":"ContainerStarted","Data":"33a4dd61152e002d06a7d3437a4a4404ff164beb24ee5327c105984b09874757"} Jul 24 00:11:14.366305 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.366235 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m2mlt" event={"ID":"354dff89-24c0-40e7-80ef-7d3c86257772","Type":"ContainerStarted","Data":"c3917a1c91c29c491bee208e6786cf0fd61c231ae810fda0f302569f9928ace2"} Jul 24 00:11:14.379108 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.379019 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal" event={"ID":"71251ac2728c24e732a941d31282182a","Type":"ContainerStarted","Data":"645bf747a6230d0e4a60519d595189b2a683433ed453959bc125b22bd117034e"} Jul 24 00:11:14.389156 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.389053 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kfmld" event={"ID":"87ccd406-4359-435e-8c88-9e677b0da0ec","Type":"ContainerStarted","Data":"591490dae06b36641346ffe719e081493ac6c9a1c6ba1bb87bdffe1df0a363f1"} Jul 24 00:11:14.395064 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.395005 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" event={"ID":"08ddb057-b851-403c-84e8-f2c50818ceaf","Type":"ContainerStarted","Data":"758d8ede9e7a5e483402f0ee08991636919467fab35d97f751d135e652c25997"} Jul 24 00:11:14.400886 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.400846 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-82.ec2.internal" event={"ID":"dd7dabc7b022df6b0fbb3ed250311918","Type":"ContainerStarted","Data":"2f1275f5f0ff1d39955704fa1ed6f0a1f726422f407e88be4f5d095ba2251fb2"} Jul 24 00:11:14.403970 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.403909 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" event={"ID":"95e1aec3-8f32-47c9-9854-c63562733dbe","Type":"ContainerStarted","Data":"e74a8c5718a38259daee126c4164c4213fd87004c0fc2742a79197ecf3b53779"} Jul 24 00:11:14.414756 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:14.414682 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:787925288f5c2c026e0cbada3e6706c38be466bf3995c478d54c10defc389780\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d1d940b941cc3129d8c30028e2371b5741771afc1860c872c66cdc8ebda6c0fe\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2e863246db3b0816c2d1279b3d1a2ee5606624d3e1c7601c715ddc7309e7b68c\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" podUID="95e1aec3-8f32-47c9-9854-c63562733dbe" Jul 24 00:11:14.415628 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.415596 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvwnr" event={"ID":"e6bdd868-f7ad-4866-965a-d60339268678","Type":"ContainerStarted","Data":"ff730db1e6cf0c7bd0b1ee26b6c87bf63c708c0e3ff09b9ce1726d377e4d1edf"} Jul 24 00:11:14.424587 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.424516 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wb5s8" event={"ID":"15937a1b-bb3c-46a5-8b30-3d2e4c260be3","Type":"ContainerStarted","Data":"bc27ad98b570e14246b066a3f514adbd7a8254de96551d2760bea5a4e68bc142"} Jul 24 00:11:14.433526 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.433491 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6hw68" event={"ID":"e6b5ccef-2d64-4b60-833b-987d428e4856","Type":"ContainerStarted","Data":"afe74adca257bf381384b9b35124df97fade59d17f45ba886d5ae590a54388fc"} Jul 24 00:11:14.444654 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.444617 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xqhjk" event={"ID":"ff1028bf-22cb-4b54-9539-ab4ed63382fc","Type":"ContainerStarted","Data":"2fe31821ad52fc0a3935a92dcfcb83a06a95a3d61b2d700151822c0a4a5f3547"} Jul 24 00:11:14.452891 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.452854 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs\") pod \"network-metrics-daemon-lqb6d\" (UID: \"3266eb6f-f15e-4b80-ba25-b0f9807214e6\") " pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:14.453061 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.452933 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:14.453130 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:14.453088 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 00:11:14.453185 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:14.453156 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret podName:ff0d6407-0175-4c0e-a54c-528dae80387a nodeName:}" failed. No retries permitted until 2026-07-24 00:11:16.453135331 +0000 UTC m=+5.058449644 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret") pod "global-pull-secret-syncer-k66hc" (UID: "ff0d6407-0175-4c0e-a54c-528dae80387a") : object "kube-system"/"original-pull-secret" not registered Jul 24 00:11:14.453330 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:14.453232 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 00:11:14.453330 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:14.453270 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs podName:3266eb6f-f15e-4b80-ba25-b0f9807214e6 nodeName:}" failed. No retries permitted until 2026-07-24 00:11:16.45325773 +0000 UTC m=+5.058572027 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs") pod "network-metrics-daemon-lqb6d" (UID: "3266eb6f-f15e-4b80-ba25-b0f9807214e6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 00:11:14.554349 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:14.554239 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cthx\" (UniqueName: \"kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx\") pod \"network-check-target-2rr2t\" (UID: \"0b1308db-61d6-49a8-b546-cf5bff91afb5\") " pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:14.554578 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:14.554439 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 00:11:14.554578 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:14.554462 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 00:11:14.554578 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:14.554475 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4cthx for pod openshift-network-diagnostics/network-check-target-2rr2t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 00:11:14.554578 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:14.554543 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx podName:0b1308db-61d6-49a8-b546-cf5bff91afb5 nodeName:}" failed. No retries permitted until 2026-07-24 00:11:16.554522032 +0000 UTC m=+5.159836349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4cthx" (UniqueName: "kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx") pod "network-check-target-2rr2t" (UID: "0b1308db-61d6-49a8-b546-cf5bff91afb5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 00:11:15.092293 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:15.092203 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 00:06:13 +0000 UTC" deadline="2028-04-29 14:38:13.934656998 +0000 UTC" Jul 24 00:11:15.092293 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:15.092250 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15494h26m58.842411768s" Jul 24 00:11:15.283630 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:15.283401 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 00:11:16.342079 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:16.342043 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:16.342612 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:16.342044 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:16.342612 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:16.342209 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:16.342612 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:16.342338 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:16.342612 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:16.342390 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:16.342612 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:16.342466 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:16.474094 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:16.473536 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs\") pod \"network-metrics-daemon-lqb6d\" (UID: \"3266eb6f-f15e-4b80-ba25-b0f9807214e6\") " pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:16.474094 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:16.473601 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:16.474094 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:16.473786 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 00:11:16.474094 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:16.473853 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret podName:ff0d6407-0175-4c0e-a54c-528dae80387a nodeName:}" failed. No retries permitted until 2026-07-24 00:11:20.473833471 +0000 UTC m=+9.079147769 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret") pod "global-pull-secret-syncer-k66hc" (UID: "ff0d6407-0175-4c0e-a54c-528dae80387a") : object "kube-system"/"original-pull-secret" not registered Jul 24 00:11:16.474094 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:16.473928 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 00:11:16.474094 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:16.474001 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs podName:3266eb6f-f15e-4b80-ba25-b0f9807214e6 nodeName:}" failed. No retries permitted until 2026-07-24 00:11:20.473987884 +0000 UTC m=+9.079302182 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs") pod "network-metrics-daemon-lqb6d" (UID: "3266eb6f-f15e-4b80-ba25-b0f9807214e6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 00:11:16.574193 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:16.574154 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cthx\" (UniqueName: \"kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx\") pod \"network-check-target-2rr2t\" (UID: \"0b1308db-61d6-49a8-b546-cf5bff91afb5\") " pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:16.574408 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:16.574391 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 00:11:16.574480 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:16.574417 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 00:11:16.574480 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:16.574431 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4cthx for pod openshift-network-diagnostics/network-check-target-2rr2t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 00:11:16.574597 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:16.574492 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx podName:0b1308db-61d6-49a8-b546-cf5bff91afb5 nodeName:}" failed. No retries permitted until 2026-07-24 00:11:20.574471655 +0000 UTC m=+9.179785966 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4cthx" (UniqueName: "kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx") pod "network-check-target-2rr2t" (UID: "0b1308db-61d6-49a8-b546-cf5bff91afb5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 00:11:18.342056 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:18.341968 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:18.342603 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:18.342161 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:18.342603 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:18.342177 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:18.342603 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:18.342402 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:18.342603 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:18.342466 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:18.342603 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:18.342585 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:20.342995 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:20.342448 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:20.342995 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:20.342578 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:20.342995 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:20.342602 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:20.342995 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:20.342683 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:20.342995 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:20.342722 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:20.342995 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:20.342792 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:20.509257 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:20.509223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs\") pod \"network-metrics-daemon-lqb6d\" (UID: \"3266eb6f-f15e-4b80-ba25-b0f9807214e6\") " pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:20.509447 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:20.509279 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:20.509447 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:20.509405 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 00:11:20.509447 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:20.509410 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 00:11:20.509615 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:20.509478 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret podName:ff0d6407-0175-4c0e-a54c-528dae80387a nodeName:}" failed. No retries permitted until 2026-07-24 00:11:28.509458442 +0000 UTC m=+17.114772749 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret") pod "global-pull-secret-syncer-k66hc" (UID: "ff0d6407-0175-4c0e-a54c-528dae80387a") : object "kube-system"/"original-pull-secret" not registered Jul 24 00:11:20.509615 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:20.509498 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs podName:3266eb6f-f15e-4b80-ba25-b0f9807214e6 nodeName:}" failed. No retries permitted until 2026-07-24 00:11:28.509489424 +0000 UTC m=+17.114803718 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs") pod "network-metrics-daemon-lqb6d" (UID: "3266eb6f-f15e-4b80-ba25-b0f9807214e6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 00:11:20.610769 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:20.610667 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cthx\" (UniqueName: \"kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx\") pod \"network-check-target-2rr2t\" (UID: \"0b1308db-61d6-49a8-b546-cf5bff91afb5\") " pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:20.610939 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:20.610853 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 00:11:20.610939 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:20.610879 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 00:11:20.610939 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:20.610891 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4cthx for pod openshift-network-diagnostics/network-check-target-2rr2t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 00:11:20.611094 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:20.610957 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx podName:0b1308db-61d6-49a8-b546-cf5bff91afb5 nodeName:}" failed. No retries permitted until 2026-07-24 00:11:28.610937985 +0000 UTC m=+17.216252290 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4cthx" (UniqueName: "kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx") pod "network-check-target-2rr2t" (UID: "0b1308db-61d6-49a8-b546-cf5bff91afb5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 00:11:22.342694 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:22.342632 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:22.343168 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:22.342837 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:22.343168 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:22.342925 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:22.343168 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:22.343021 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:22.343446 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:22.343406 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:22.343446 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:22.343416 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:24.342420 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:24.342383 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:24.342918 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:24.342391 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:24.342918 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:24.342530 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:24.342918 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:24.342559 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:24.342918 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:24.342695 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:24.342918 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:24.342797 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:26.341751 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:26.341714 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:26.342237 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:26.341716 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:26.342237 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:26.341863 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:26.342237 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:26.341714 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:26.342237 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:26.341934 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:26.342237 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:26.342011 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:28.342418 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:28.342384 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:28.342903 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:28.342384 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:28.342903 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:28.342536 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:28.342903 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:28.342392 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:28.342903 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:28.342614 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:28.342903 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:28.342676 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:28.574334 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:28.574287 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs\") pod \"network-metrics-daemon-lqb6d\" (UID: \"3266eb6f-f15e-4b80-ba25-b0f9807214e6\") " pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:28.574535 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:28.574352 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:28.574535 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:28.574454 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 00:11:28.574668 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:28.574541 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs podName:3266eb6f-f15e-4b80-ba25-b0f9807214e6 nodeName:}" failed. No retries permitted until 2026-07-24 00:11:44.574523232 +0000 UTC m=+33.179837525 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs") pod "network-metrics-daemon-lqb6d" (UID: "3266eb6f-f15e-4b80-ba25-b0f9807214e6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 00:11:28.574668 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:28.574465 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 00:11:28.574668 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:28.574598 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret podName:ff0d6407-0175-4c0e-a54c-528dae80387a nodeName:}" failed. No retries permitted until 2026-07-24 00:11:44.574586842 +0000 UTC m=+33.179901144 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret") pod "global-pull-secret-syncer-k66hc" (UID: "ff0d6407-0175-4c0e-a54c-528dae80387a") : object "kube-system"/"original-pull-secret" not registered Jul 24 00:11:28.675756 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:28.675655 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cthx\" (UniqueName: \"kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx\") pod \"network-check-target-2rr2t\" (UID: \"0b1308db-61d6-49a8-b546-cf5bff91afb5\") " pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:28.675928 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:28.675777 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 00:11:28.675928 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:28.675803 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 00:11:28.675928 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:28.675815 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4cthx for pod openshift-network-diagnostics/network-check-target-2rr2t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 00:11:28.675928 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:28.675876 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx podName:0b1308db-61d6-49a8-b546-cf5bff91afb5 nodeName:}" failed. No retries permitted until 2026-07-24 00:11:44.675859658 +0000 UTC m=+33.281173952 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4cthx" (UniqueName: "kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx") pod "network-check-target-2rr2t" (UID: "0b1308db-61d6-49a8-b546-cf5bff91afb5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 00:11:30.342788 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:30.342548 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:30.343307 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:30.342548 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:30.343307 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:30.342901 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:30.343307 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:30.342983 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:30.343307 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:30.342578 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:30.343307 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:30.343081 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:32.342620 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:32.342582 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:32.343126 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:32.342708 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:32.343126 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:32.342716 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:32.343126 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:32.342734 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:32.343126 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:32.342838 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:32.343126 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:32.342931 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:33.501556 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.501277 2560 generic.go:356] "Generic (PLEG): container finished" podID="e6bdd868-f7ad-4866-965a-d60339268678" containerID="8cd20f0bac88ad55fce0b9afb89485cdb1aafc09653b12574cd4d12eb89391c4" exitCode=0 Jul 24 00:11:33.502569 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.501360 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvwnr" event={"ID":"e6bdd868-f7ad-4866-965a-d60339268678","Type":"ContainerDied","Data":"8cd20f0bac88ad55fce0b9afb89485cdb1aafc09653b12574cd4d12eb89391c4"} Jul 24 00:11:33.503971 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.503913 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wb5s8" event={"ID":"15937a1b-bb3c-46a5-8b30-3d2e4c260be3","Type":"ContainerStarted","Data":"1bce92be0975db418b34b52f890af9ea38df01784508844602a032e08ca9dec9"} Jul 24 00:11:33.505861 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.505829 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6hw68" event={"ID":"e6b5ccef-2d64-4b60-833b-987d428e4856","Type":"ContainerStarted","Data":"25a05a63dd3b4ccf819d6b9f8aca0959c6a10759cd379485eeefc42623735512"} Jul 24 00:11:33.507919 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.507632 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c4c45" event={"ID":"567139c0-866a-4ff0-9085-579046198698","Type":"ContainerStarted","Data":"5b132f51a69987c8d8cc435101523b905ce1b955078bbbf2ad03156578203e8c"} Jul 24 00:11:33.509454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.509418 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m2mlt" event={"ID":"354dff89-24c0-40e7-80ef-7d3c86257772","Type":"ContainerStarted","Data":"fc371b1b234879758df236fb92c5425421de0539c21be16a3da40f992e1600cf"} Jul 24 00:11:33.511195 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.511157 2560 generic.go:356] "Generic (PLEG): container finished" podID="71251ac2728c24e732a941d31282182a" containerID="262df8c4edd97862cbdd6fe4b88a78ada7d9b9b36b9d56ce44c67c4188b59242" exitCode=0 Jul 24 00:11:33.511327 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.511243 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal" event={"ID":"71251ac2728c24e732a941d31282182a","Type":"ContainerDied","Data":"262df8c4edd97862cbdd6fe4b88a78ada7d9b9b36b9d56ce44c67c4188b59242"} Jul 24 00:11:33.513002 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.512971 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kfmld" event={"ID":"87ccd406-4359-435e-8c88-9e677b0da0ec","Type":"ContainerStarted","Data":"4a36010fac779a5aaf3bda216711a92de962f330de5e08cc109dee13e499864c"} Jul 24 00:11:33.515717 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.515687 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" event={"ID":"08ddb057-b851-403c-84e8-f2c50818ceaf","Type":"ContainerStarted","Data":"d3906a51ddfcbe9297713272b6cc07281f3a0ba4ac0dac015c64405ded5e2643"} Jul 24 00:11:33.515866 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.515764 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" event={"ID":"08ddb057-b851-403c-84e8-f2c50818ceaf","Type":"ContainerStarted","Data":"14718a9a1c41c64786f14fa5bd2808479afb31ada9403daba00811503ed4aa94"} Jul 24 00:11:33.515866 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.515845 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" event={"ID":"08ddb057-b851-403c-84e8-f2c50818ceaf","Type":"ContainerStarted","Data":"e8c9b1e54bd735129a6f805a72f48f991d66125e298cb655419f733fbfb4c495"} Jul 24 00:11:33.517491 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.517468 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-82.ec2.internal" event={"ID":"dd7dabc7b022df6b0fbb3ed250311918","Type":"ContainerStarted","Data":"073d7b7c3745d843715159075fff5afa129b946e401bc97ae9450ec5de4097bd"} Jul 24 00:11:33.547593 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.547402 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-m2mlt" podStartSLOduration=2.028972559 podStartE2EDuration="21.547379961s" totalImagesPullingTime="19.518407402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 00:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:11:33.546971767 +0000 UTC m=+22.152286096" watchObservedRunningTime="2026-07-24 00:11:33.547379961 +0000 UTC m=+22.152694275" Jul 24 00:11:33.567672 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.565285 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-c4c45" podStartSLOduration=1.153148008 podStartE2EDuration="20.565262232s" totalImagesPullingTime="19.412114224s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 00:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:11:33.564773778 +0000 UTC m=+22.170088095" watchObservedRunningTime="2026-07-24 00:11:33.565262232 +0000 UTC m=+22.170576548" Jul 24 00:11:33.583021 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.582845 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kfmld" podStartSLOduration=1.274610539 podStartE2EDuration="20.58281816s" totalImagesPullingTime="19.308207621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:11:33.582289765 +0000 UTC m=+22.187604082" watchObservedRunningTime="2026-07-24 00:11:33.58281816 +0000 UTC m=+22.188132477" Jul 24 00:11:33.623166 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.623111 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6hw68" podStartSLOduration=1.243404614 podStartE2EDuration="20.623093975s" totalImagesPullingTime="19.379689361s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 00:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:11:33.622013048 +0000 UTC m=+22.227327365" watchObservedRunningTime="2026-07-24 00:11:33.623093975 +0000 UTC m=+22.228408290" Jul 24 00:11:33.638976 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.638918 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wb5s8" podStartSLOduration=1.293238745 podStartE2EDuration="20.638894317s" totalImagesPullingTime="19.345655572s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 00:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:11:33.638488981 +0000 UTC m=+22.243803299" watchObservedRunningTime="2026-07-24 00:11:33.638894317 +0000 UTC m=+22.244208634" Jul 24 00:11:33.653406 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:33.653352 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-82.ec2.internal" podStartSLOduration=20.653332799 podStartE2EDuration="20.653332799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 00:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:11:33.653097138 +0000 UTC m=+22.258411453" watchObservedRunningTime="2026-07-24 00:11:33.653332799 +0000 UTC m=+22.258647160" Jul 24 00:11:34.342083 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:34.341751 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:34.342083 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:34.341751 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:34.342316 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:34.342107 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:34.342316 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:34.342193 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:34.342316 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:34.341751 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:34.342316 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:34.342287 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:34.522217 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:34.522176 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal" event={"ID":"71251ac2728c24e732a941d31282182a","Type":"ContainerStarted","Data":"e53d51b3661eef4466f408effdd43b76facadd11b5d98ea7357d72d2a1167344"} Jul 24 00:11:34.526655 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:34.526605 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" event={"ID":"08ddb057-b851-403c-84e8-f2c50818ceaf","Type":"ContainerStarted","Data":"d07a99ae9c30a8f6e2ae3680c2cf32bfa73e5fb09125d15c0a566d5bbb596be1"} Jul 24 00:11:34.526844 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:34.526716 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" event={"ID":"08ddb057-b851-403c-84e8-f2c50818ceaf","Type":"ContainerStarted","Data":"f11b49d9caa2ec3a14a1b3a72b274fe506910d87670e63371e6f27c045add5f6"} Jul 24 00:11:34.526844 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:34.526768 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" event={"ID":"08ddb057-b851-403c-84e8-f2c50818ceaf","Type":"ContainerStarted","Data":"f282dcb2cbb858b5523d374e6f956f05fbb27758af8c7325fe9c3cb32e6f9e24"} Jul 24 00:11:34.539709 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:34.539511 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-82.ec2.internal" podStartSLOduration=21.539488077 podStartE2EDuration="21.539488077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 00:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:11:34.539212282 +0000 UTC m=+23.144526601" watchObservedRunningTime="2026-07-24 00:11:34.539488077 +0000 UTC m=+23.144802393" Jul 24 00:11:35.451069 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:35.450984 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:35.530476 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:35.530434 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" event={"ID":"95e1aec3-8f32-47c9-9854-c63562733dbe","Type":"ContainerStarted","Data":"b7c7d5f1f4ffba16473657818dc358706d088d5bd1d5d3550870454bbbda65b5"} Jul 24 00:11:35.532123 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:35.532091 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xqhjk" event={"ID":"ff1028bf-22cb-4b54-9539-ab4ed63382fc","Type":"ContainerStarted","Data":"65706077f6453cc886e23d6bf032b47d1b3eb22217ddd6b88b114b047dcec5f9"} Jul 24 00:11:35.556363 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:35.556321 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:35.701755 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:35.701672 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:35.799330 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:35.799286 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:36.342012 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:36.341973 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:36.342288 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:36.341988 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:36.342288 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:36.342105 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:36.342288 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:36.341988 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:36.342288 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:36.342192 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:36.342288 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:36.342264 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:36.415782 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:36.415740 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:36.538593 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:36.538516 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" event={"ID":"08ddb057-b851-403c-84e8-f2c50818ceaf","Type":"ContainerStarted","Data":"7207d942a597890cecac794cda44c24e14c1c2b437ac3303a275250cb7bb1291"} Jul 24 00:11:36.755513 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:36.755491 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:36.850987 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:36.850961 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 00:11:36.853697 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:36.853670 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:37.142229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:37.142004 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T00:11:36.850983958Z","UUID":"aa1d0b96-47a2-4492-94d5-74cd7d325955","Handler":null,"Name":"","Endpoint":""} Jul 24 00:11:37.143699 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:37.143681 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 00:11:37.143820 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:37.143718 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 00:11:37.506608 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:37.506557 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kfmld" Jul 24 00:11:37.507198 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:37.507179 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kfmld" Jul 24 00:11:37.523994 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:37.523936 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xqhjk" podStartSLOduration=5.096374573 podStartE2EDuration="24.523911054s" totalImagesPullingTime="19.427536481s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 00:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:11:35.551967992 +0000 UTC m=+24.157282305" watchObservedRunningTime="2026-07-24 00:11:37.523911054 +0000 UTC m=+26.129225373" Jul 24 00:11:37.542506 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:37.542460 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" event={"ID":"95e1aec3-8f32-47c9-9854-c63562733dbe","Type":"ContainerStarted","Data":"33b4feb549a72d8a96a614ca3a051f8417d9b84e4a728b6f2f4f866d07d5c3c1"} Jul 24 00:11:37.543050 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:37.542715 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kfmld" Jul 24 00:11:37.543261 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:37.543245 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kfmld" Jul 24 00:11:37.615041 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:37.615005 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:38.342220 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:38.342135 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:38.342220 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:38.342185 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:38.342434 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:38.342143 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:38.342434 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:38.342289 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:38.342434 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:38.342360 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:38.342591 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:38.342437 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:39.079752 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:39.079715 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:39.136680 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:39.136629 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:39.686199 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:39.686166 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:39.749281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:39.749250 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:40.341745 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:40.341566 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:40.342513 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:40.341583 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:40.342513 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:40.341812 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:40.342513 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:40.341595 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:40.342513 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:40.341908 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:40.342513 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:40.341968 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:40.413859 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:40.413827 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:40.512921 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:40.512891 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:40.552729 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:40.552696 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" event={"ID":"08ddb057-b851-403c-84e8-f2c50818ceaf","Type":"ContainerStarted","Data":"6f15786d62524a8e36332c5fc091e2ee7a2633098cf165ce2800a95b39743af5"} Jul 24 00:11:40.552953 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:40.552935 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:40.554692 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:40.554663 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" event={"ID":"95e1aec3-8f32-47c9-9854-c63562733dbe","Type":"ContainerStarted","Data":"fd8b9830ea5ec00c6ca067b1beb0bdffc17a4835ef72f50cc105d3c55683061d"} Jul 24 00:11:40.556343 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:40.556321 2560 generic.go:356] "Generic (PLEG): container finished" podID="e6bdd868-f7ad-4866-965a-d60339268678" containerID="997650049ed3b0a8809f0b8d36592daac7077bc81557e1453623262c1b35a827" exitCode=0 Jul 24 00:11:40.556425 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:40.556358 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvwnr" event={"ID":"e6bdd868-f7ad-4866-965a-d60339268678","Type":"ContainerDied","Data":"997650049ed3b0a8809f0b8d36592daac7077bc81557e1453623262c1b35a827"} Jul 24 00:11:40.568750 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:40.568731 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:40.581321 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:40.581273 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" podStartSLOduration=9.189682572 podStartE2EDuration="28.581260495s" totalImagesPullingTime="19.391577923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 00:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:11:40.580772184 +0000 UTC m=+29.186086501" watchObservedRunningTime="2026-07-24 00:11:40.581260495 +0000 UTC m=+29.186574810" Jul 24 00:11:40.594986 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:40.594959 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:40.662465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:40.662420 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c4hx" podStartSLOduration=2.463697174 podStartE2EDuration="28.66240616s" totalImagesPullingTime="26.198708986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 00:11:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=3 observedRunningTime="2026-07-24 00:11:40.601342575 +0000 UTC m=+29.206656891" watchObservedRunningTime="2026-07-24 00:11:40.66240616 +0000 UTC m=+29.267720540" Jul 24 00:11:41.535857 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:41.535827 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:41.561450 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:41.561293 2560 generic.go:356] "Generic (PLEG): container finished" podID="e6bdd868-f7ad-4866-965a-d60339268678" containerID="c42655698fc24cce4cc55d53038d27f52c4cb7f698ea44efd360cd116588de23" exitCode=0 Jul 24 00:11:41.561587 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:41.561354 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvwnr" event={"ID":"e6bdd868-f7ad-4866-965a-d60339268678","Type":"ContainerDied","Data":"c42655698fc24cce4cc55d53038d27f52c4cb7f698ea44efd360cd116588de23"} Jul 24 00:11:41.562236 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:41.562221 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:41.562365 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:41.562302 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:41.577384 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:41.577357 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:11:42.022069 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:42.022040 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2rr2t"] Jul 24 00:11:42.022275 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:42.022184 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:42.022345 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:42.022276 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:42.025392 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:42.025367 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k66hc"] Jul 24 00:11:42.025541 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:42.025476 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:42.025664 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:42.025610 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:42.026031 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:42.025996 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lqb6d"] Jul 24 00:11:42.026138 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:42.026094 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:42.026207 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:42.026183 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:42.566526 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:42.566493 2560 generic.go:356] "Generic (PLEG): container finished" podID="e6bdd868-f7ad-4866-965a-d60339268678" containerID="4dc6eab60762e4287500c45598f1936f0711f6da6fd8cbadf1e9a047d4b149e0" exitCode=0 Jul 24 00:11:42.567020 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:42.566563 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvwnr" event={"ID":"e6bdd868-f7ad-4866-965a-d60339268678","Type":"ContainerDied","Data":"4dc6eab60762e4287500c45598f1936f0711f6da6fd8cbadf1e9a047d4b149e0"} Jul 24 00:11:43.341788 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:43.341750 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:43.341959 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:43.341892 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:43.341959 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:43.341938 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:43.342086 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:43.342033 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:44.342412 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:44.342375 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:44.342882 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:44.342516 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rr2t" podUID="0b1308db-61d6-49a8-b546-cf5bff91afb5" Jul 24 00:11:44.592630 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:44.592551 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs\") pod \"network-metrics-daemon-lqb6d\" (UID: \"3266eb6f-f15e-4b80-ba25-b0f9807214e6\") " pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:44.592630 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:44.592588 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:44.592830 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:44.592698 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 00:11:44.592830 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:44.592702 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 00:11:44.592830 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:44.592748 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret podName:ff0d6407-0175-4c0e-a54c-528dae80387a nodeName:}" failed. No retries permitted until 2026-07-24 00:12:16.592735877 +0000 UTC m=+65.198050172 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret") pod "global-pull-secret-syncer-k66hc" (UID: "ff0d6407-0175-4c0e-a54c-528dae80387a") : object "kube-system"/"original-pull-secret" not registered Jul 24 00:11:44.592830 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:44.592762 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs podName:3266eb6f-f15e-4b80-ba25-b0f9807214e6 nodeName:}" failed. No retries permitted until 2026-07-24 00:12:16.592756256 +0000 UTC m=+65.198070549 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs") pod "network-metrics-daemon-lqb6d" (UID: "3266eb6f-f15e-4b80-ba25-b0f9807214e6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 00:11:44.693560 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:44.693516 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cthx\" (UniqueName: \"kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx\") pod \"network-check-target-2rr2t\" (UID: \"0b1308db-61d6-49a8-b546-cf5bff91afb5\") " pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:44.693744 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:44.693704 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 00:11:44.693744 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:44.693728 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 00:11:44.693744 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:44.693739 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4cthx for pod openshift-network-diagnostics/network-check-target-2rr2t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 00:11:44.693855 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:44.693801 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx podName:0b1308db-61d6-49a8-b546-cf5bff91afb5 nodeName:}" failed. No retries permitted until 2026-07-24 00:12:16.693780772 +0000 UTC m=+65.299095075 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4cthx" (UniqueName: "kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx") pod "network-check-target-2rr2t" (UID: "0b1308db-61d6-49a8-b546-cf5bff91afb5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 00:11:45.341962 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.341926 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:45.342144 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.341928 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:45.342144 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:45.342071 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k66hc" podUID="ff0d6407-0175-4c0e-a54c-528dae80387a" Jul 24 00:11:45.342256 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:11:45.342168 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqb6d" podUID="3266eb6f-f15e-4b80-ba25-b0f9807214e6" Jul 24 00:11:45.435278 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.435246 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:45.499971 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.499936 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:45.696830 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.696806 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-82.ec2.internal" event="NodeReady" Jul 24 00:11:45.696984 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.696956 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 00:11:45.750357 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.750324 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fxm4z"] Jul 24 00:11:45.752552 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.752511 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fxm4z" Jul 24 00:11:45.753735 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.753716 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-d7q6r"] Jul 24 00:11:45.755352 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.755334 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dcszn\"" Jul 24 00:11:45.755447 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.755376 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:45.755614 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.755595 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 00:11:45.755685 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.755620 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 00:11:45.757928 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.757913 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nsh42\"" Jul 24 00:11:45.758336 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.758303 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 00:11:45.758425 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.758347 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 00:11:45.758425 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.758364 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 00:11:45.758574 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.758367 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 00:11:45.766388 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.766342 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fxm4z"] Jul 24 00:11:45.768359 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.768315 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d7q6r"] Jul 24 00:11:45.858528 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.858440 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rsb8q"] Jul 24 00:11:45.861106 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.861081 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-p88lj"] Jul 24 00:11:45.861259 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.861240 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rsb8q" Jul 24 00:11:45.863047 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.863027 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-p88lj" Jul 24 00:11:45.866325 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.866301 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qlhxg\"" Jul 24 00:11:45.866419 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.866337 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 00:11:45.866485 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.866305 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 00:11:45.866628 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.866613 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-k8xbz\"" Jul 24 00:11:45.866709 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.866686 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 24 00:11:45.866800 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.866784 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 00:11:45.866906 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.866889 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 24 00:11:45.867721 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.867697 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:45.873445 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.873401 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-p88lj"] Jul 24 00:11:45.877774 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.877727 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rsb8q"] Jul 24 00:11:45.902546 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.902521 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vh6q\" (UniqueName: \"kubernetes.io/projected/989c9067-460b-4838-b1b0-3da75e2a848f-kube-api-access-7vh6q\") pod \"dns-default-fxm4z\" (UID: \"989c9067-460b-4838-b1b0-3da75e2a848f\") " pod="openshift-dns/dns-default-fxm4z" Jul 24 00:11:45.902736 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.902556 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/989c9067-460b-4838-b1b0-3da75e2a848f-tmp-dir\") pod \"dns-default-fxm4z\" (UID: \"989c9067-460b-4838-b1b0-3da75e2a848f\") " pod="openshift-dns/dns-default-fxm4z" Jul 24 00:11:45.902736 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.902583 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6prg\" (UniqueName: \"kubernetes.io/projected/1a539eef-502e-48e6-8020-2e6d9cc6ff5b-kube-api-access-f6prg\") pod \"insights-runtime-extractor-d7q6r\" (UID: \"1a539eef-502e-48e6-8020-2e6d9cc6ff5b\") " pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:45.902736 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.902633 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/989c9067-460b-4838-b1b0-3da75e2a848f-config-volume\") pod \"dns-default-fxm4z\" (UID: \"989c9067-460b-4838-b1b0-3da75e2a848f\") " pod="openshift-dns/dns-default-fxm4z" Jul 24 00:11:45.902736 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.902692 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a539eef-502e-48e6-8020-2e6d9cc6ff5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d7q6r\" (UID: \"1a539eef-502e-48e6-8020-2e6d9cc6ff5b\") " pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:45.902736 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.902726 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a539eef-502e-48e6-8020-2e6d9cc6ff5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d7q6r\" (UID: \"1a539eef-502e-48e6-8020-2e6d9cc6ff5b\") " pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:45.902928 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.902815 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/989c9067-460b-4838-b1b0-3da75e2a848f-metrics-tls\") pod \"dns-default-fxm4z\" (UID: \"989c9067-460b-4838-b1b0-3da75e2a848f\") " pod="openshift-dns/dns-default-fxm4z" Jul 24 00:11:45.902928 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.902842 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1a539eef-502e-48e6-8020-2e6d9cc6ff5b-crio-socket\") pod \"insights-runtime-extractor-d7q6r\" (UID: \"1a539eef-502e-48e6-8020-2e6d9cc6ff5b\") " pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:45.902928 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.902862 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a539eef-502e-48e6-8020-2e6d9cc6ff5b-data-volume\") pod \"insights-runtime-extractor-d7q6r\" (UID: \"1a539eef-502e-48e6-8020-2e6d9cc6ff5b\") " pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:45.941029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:45.940994 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:46.004104 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.003925 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a539eef-502e-48e6-8020-2e6d9cc6ff5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d7q6r\" (UID: \"1a539eef-502e-48e6-8020-2e6d9cc6ff5b\") " pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:46.004282 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.004119 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94rqs\" (UniqueName: \"kubernetes.io/projected/5ff612cc-1d5c-4195-b649-be35a283ce97-kube-api-access-94rqs\") pod \"downloads-7c97dc4555-p88lj\" (UID: \"5ff612cc-1d5c-4195-b649-be35a283ce97\") " pod="openshift-console/downloads-7c97dc4555-p88lj" Jul 24 00:11:46.004282 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.004146 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a539eef-502e-48e6-8020-2e6d9cc6ff5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d7q6r\" (UID: \"1a539eef-502e-48e6-8020-2e6d9cc6ff5b\") " pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:46.004282 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.004184 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/989c9067-460b-4838-b1b0-3da75e2a848f-metrics-tls\") pod \"dns-default-fxm4z\" (UID: \"989c9067-460b-4838-b1b0-3da75e2a848f\") " pod="openshift-dns/dns-default-fxm4z" Jul 24 00:11:46.004282 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.004210 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1a539eef-502e-48e6-8020-2e6d9cc6ff5b-crio-socket\") pod \"insights-runtime-extractor-d7q6r\" (UID: \"1a539eef-502e-48e6-8020-2e6d9cc6ff5b\") " pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:46.004282 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.004237 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a539eef-502e-48e6-8020-2e6d9cc6ff5b-data-volume\") pod \"insights-runtime-extractor-d7q6r\" (UID: \"1a539eef-502e-48e6-8020-2e6d9cc6ff5b\") " pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:46.004496 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.004429 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c87883b-9a82-4049-a788-b28dc9f49340-cert\") pod \"ingress-canary-rsb8q\" (UID: \"0c87883b-9a82-4049-a788-b28dc9f49340\") " pod="openshift-ingress-canary/ingress-canary-rsb8q" Jul 24 00:11:46.004496 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.004466 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zffr7\" (UniqueName: \"kubernetes.io/projected/0c87883b-9a82-4049-a788-b28dc9f49340-kube-api-access-zffr7\") pod \"ingress-canary-rsb8q\" (UID: \"0c87883b-9a82-4049-a788-b28dc9f49340\") " pod="openshift-ingress-canary/ingress-canary-rsb8q" Jul 24 00:11:46.004496 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.004485 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1a539eef-502e-48e6-8020-2e6d9cc6ff5b-crio-socket\") pod \"insights-runtime-extractor-d7q6r\" (UID: \"1a539eef-502e-48e6-8020-2e6d9cc6ff5b\") " pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:46.004631 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.004539 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vh6q\" (UniqueName: \"kubernetes.io/projected/989c9067-460b-4838-b1b0-3da75e2a848f-kube-api-access-7vh6q\") pod \"dns-default-fxm4z\" (UID: \"989c9067-460b-4838-b1b0-3da75e2a848f\") " pod="openshift-dns/dns-default-fxm4z" Jul 24 00:11:46.004631 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.004540 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a539eef-502e-48e6-8020-2e6d9cc6ff5b-data-volume\") pod \"insights-runtime-extractor-d7q6r\" (UID: \"1a539eef-502e-48e6-8020-2e6d9cc6ff5b\") " pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:46.004631 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.004577 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/989c9067-460b-4838-b1b0-3da75e2a848f-tmp-dir\") pod \"dns-default-fxm4z\" (UID: \"989c9067-460b-4838-b1b0-3da75e2a848f\") " pod="openshift-dns/dns-default-fxm4z" Jul 24 00:11:46.004631 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.004604 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6prg\" (UniqueName: \"kubernetes.io/projected/1a539eef-502e-48e6-8020-2e6d9cc6ff5b-kube-api-access-f6prg\") pod \"insights-runtime-extractor-d7q6r\" (UID: \"1a539eef-502e-48e6-8020-2e6d9cc6ff5b\") " pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:46.004818 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.004675 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/989c9067-460b-4838-b1b0-3da75e2a848f-config-volume\") pod \"dns-default-fxm4z\" (UID: \"989c9067-460b-4838-b1b0-3da75e2a848f\") " pod="openshift-dns/dns-default-fxm4z" Jul 24 00:11:46.004818 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.004779 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a539eef-502e-48e6-8020-2e6d9cc6ff5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d7q6r\" (UID: \"1a539eef-502e-48e6-8020-2e6d9cc6ff5b\") " pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:46.004940 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.004857 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/989c9067-460b-4838-b1b0-3da75e2a848f-tmp-dir\") pod \"dns-default-fxm4z\" (UID: \"989c9067-460b-4838-b1b0-3da75e2a848f\") " pod="openshift-dns/dns-default-fxm4z" Jul 24 00:11:46.005252 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.005209 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/989c9067-460b-4838-b1b0-3da75e2a848f-config-volume\") pod \"dns-default-fxm4z\" (UID: \"989c9067-460b-4838-b1b0-3da75e2a848f\") " pod="openshift-dns/dns-default-fxm4z" Jul 24 00:11:46.008296 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.008274 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a539eef-502e-48e6-8020-2e6d9cc6ff5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d7q6r\" (UID: \"1a539eef-502e-48e6-8020-2e6d9cc6ff5b\") " pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:46.008399 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.008280 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/989c9067-460b-4838-b1b0-3da75e2a848f-metrics-tls\") pod \"dns-default-fxm4z\" (UID: \"989c9067-460b-4838-b1b0-3da75e2a848f\") " pod="openshift-dns/dns-default-fxm4z" Jul 24 00:11:46.013941 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.013921 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6prg\" (UniqueName: \"kubernetes.io/projected/1a539eef-502e-48e6-8020-2e6d9cc6ff5b-kube-api-access-f6prg\") pod \"insights-runtime-extractor-d7q6r\" (UID: \"1a539eef-502e-48e6-8020-2e6d9cc6ff5b\") " pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:46.014066 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.013954 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vh6q\" (UniqueName: \"kubernetes.io/projected/989c9067-460b-4838-b1b0-3da75e2a848f-kube-api-access-7vh6q\") pod \"dns-default-fxm4z\" (UID: \"989c9067-460b-4838-b1b0-3da75e2a848f\") " pod="openshift-dns/dns-default-fxm4z" Jul 24 00:11:46.042501 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.042443 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:46.063550 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.063515 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fxm4z" Jul 24 00:11:46.069215 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.069190 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d7q6r" Jul 24 00:11:46.105338 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.105304 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c87883b-9a82-4049-a788-b28dc9f49340-cert\") pod \"ingress-canary-rsb8q\" (UID: \"0c87883b-9a82-4049-a788-b28dc9f49340\") " pod="openshift-ingress-canary/ingress-canary-rsb8q" Jul 24 00:11:46.105338 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.105341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zffr7\" (UniqueName: \"kubernetes.io/projected/0c87883b-9a82-4049-a788-b28dc9f49340-kube-api-access-zffr7\") pod \"ingress-canary-rsb8q\" (UID: \"0c87883b-9a82-4049-a788-b28dc9f49340\") " pod="openshift-ingress-canary/ingress-canary-rsb8q" Jul 24 00:11:46.105503 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.105418 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94rqs\" (UniqueName: \"kubernetes.io/projected/5ff612cc-1d5c-4195-b649-be35a283ce97-kube-api-access-94rqs\") pod \"downloads-7c97dc4555-p88lj\" (UID: \"5ff612cc-1d5c-4195-b649-be35a283ce97\") " pod="openshift-console/downloads-7c97dc4555-p88lj" Jul 24 00:11:46.109308 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.109050 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c87883b-9a82-4049-a788-b28dc9f49340-cert\") pod \"ingress-canary-rsb8q\" (UID: \"0c87883b-9a82-4049-a788-b28dc9f49340\") " pod="openshift-ingress-canary/ingress-canary-rsb8q" Jul 24 00:11:46.117350 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.117322 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zffr7\" (UniqueName: \"kubernetes.io/projected/0c87883b-9a82-4049-a788-b28dc9f49340-kube-api-access-zffr7\") pod \"ingress-canary-rsb8q\" (UID: \"0c87883b-9a82-4049-a788-b28dc9f49340\") " pod="openshift-ingress-canary/ingress-canary-rsb8q" Jul 24 00:11:46.118072 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.118050 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94rqs\" (UniqueName: \"kubernetes.io/projected/5ff612cc-1d5c-4195-b649-be35a283ce97-kube-api-access-94rqs\") pod \"downloads-7c97dc4555-p88lj\" (UID: \"5ff612cc-1d5c-4195-b649-be35a283ce97\") " pod="openshift-console/downloads-7c97dc4555-p88lj" Jul 24 00:11:46.144623 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.144564 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:46.172718 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.172683 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rsb8q" Jul 24 00:11:46.179872 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.179843 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-p88lj" Jul 24 00:11:46.341704 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.341664 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:11:46.344405 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.344373 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-79cjj\"" Jul 24 00:11:46.344572 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.344404 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 00:11:46.344572 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.344416 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 00:11:46.519458 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.519391 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:46.581743 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.581710 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:46.909234 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:46.909152 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:47.012155 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.012124 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:47.341544 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.341513 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:11:47.341725 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.341683 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:11:47.345099 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.345077 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 00:11:47.345218 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.345129 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 00:11:47.345274 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.345241 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rvqg9\"" Jul 24 00:11:47.518686 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.518587 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:47.638800 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.638777 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rsb8q"] Jul 24 00:11:47.639418 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.639394 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fxm4z"] Jul 24 00:11:47.639932 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.639915 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d7q6r"] Jul 24 00:11:47.644694 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.644676 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-p88lj"] Jul 24 00:11:47.700043 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.700015 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:47.716520 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:47.716487 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c87883b_9a82_4049_a788_b28dc9f49340.slice/crio-c392db7b55d4aeebfd0fcebb340925f3af1cefb8a78ce361513c1c92c8e41494 WatchSource:0}: Error finding container c392db7b55d4aeebfd0fcebb340925f3af1cefb8a78ce361513c1c92c8e41494: Status 404 returned error can't find the container with id c392db7b55d4aeebfd0fcebb340925f3af1cefb8a78ce361513c1c92c8e41494 Jul 24 00:11:47.717094 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:47.717067 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod989c9067_460b_4838_b1b0_3da75e2a848f.slice/crio-161d056dc8953427948cab95735a0f0ecd2a4916b88751fdb32284e30d940a7a WatchSource:0}: Error finding container 161d056dc8953427948cab95735a0f0ecd2a4916b88751fdb32284e30d940a7a: Status 404 returned error can't find the container with id 161d056dc8953427948cab95735a0f0ecd2a4916b88751fdb32284e30d940a7a Jul 24 00:11:47.718471 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:47.718442 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a539eef_502e_48e6_8020_2e6d9cc6ff5b.slice/crio-f10f89e1ca861a814b656a6b1ca5a918079eb9384068295ee4367674c911d5a3 WatchSource:0}: Error finding container f10f89e1ca861a814b656a6b1ca5a918079eb9384068295ee4367674c911d5a3: Status 404 returned error can't find the container with id f10f89e1ca861a814b656a6b1ca5a918079eb9384068295ee4367674c911d5a3 Jul 24 00:11:47.718831 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:47.718791 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ff612cc_1d5c_4195_b649_be35a283ce97.slice/crio-2fea798ab87de179e59e046fd4db4a8f4b5997b1ddb86159266f74c1e14a1e74 WatchSource:0}: Error finding container 2fea798ab87de179e59e046fd4db4a8f4b5997b1ddb86159266f74c1e14a1e74: Status 404 returned error can't find the container with id 2fea798ab87de179e59e046fd4db4a8f4b5997b1ddb86159266f74c1e14a1e74 Jul 24 00:11:47.745660 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.745618 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-hmvqn"] Jul 24 00:11:47.761074 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.761054 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-hmvqn"] Jul 24 00:11:47.761220 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.761172 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" Jul 24 00:11:47.763555 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.763527 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 00:11:47.764031 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.763944 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 00:11:47.764031 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.763979 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 00:11:47.764212 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.764092 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 24 00:11:47.764212 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.764121 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-sqbzk\"" Jul 24 00:11:47.764324 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.764311 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 24 00:11:47.783618 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.783596 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:47.920622 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.920451 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8355e71-b509-4932-9797-afcaded7f8fe-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-hmvqn\" (UID: \"d8355e71-b509-4932-9797-afcaded7f8fe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" Jul 24 00:11:47.920778 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.920721 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px9gw\" (UniqueName: \"kubernetes.io/projected/d8355e71-b509-4932-9797-afcaded7f8fe-kube-api-access-px9gw\") pod \"prometheus-operator-84d69f794d-hmvqn\" (UID: \"d8355e71-b509-4932-9797-afcaded7f8fe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" Jul 24 00:11:47.920778 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.920757 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8355e71-b509-4932-9797-afcaded7f8fe-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-hmvqn\" (UID: \"d8355e71-b509-4932-9797-afcaded7f8fe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" Jul 24 00:11:47.920874 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.920783 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8355e71-b509-4932-9797-afcaded7f8fe-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-hmvqn\" (UID: \"d8355e71-b509-4932-9797-afcaded7f8fe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" Jul 24 00:11:47.929812 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.929782 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:47.986674 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:47.986651 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:48.021791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.021764 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8355e71-b509-4932-9797-afcaded7f8fe-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-hmvqn\" (UID: \"d8355e71-b509-4932-9797-afcaded7f8fe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" Jul 24 00:11:48.021933 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.021917 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px9gw\" (UniqueName: \"kubernetes.io/projected/d8355e71-b509-4932-9797-afcaded7f8fe-kube-api-access-px9gw\") pod \"prometheus-operator-84d69f794d-hmvqn\" (UID: \"d8355e71-b509-4932-9797-afcaded7f8fe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" Jul 24 00:11:48.021990 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.021938 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8355e71-b509-4932-9797-afcaded7f8fe-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-hmvqn\" (UID: \"d8355e71-b509-4932-9797-afcaded7f8fe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" Jul 24 00:11:48.021990 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.021957 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8355e71-b509-4932-9797-afcaded7f8fe-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-hmvqn\" (UID: \"d8355e71-b509-4932-9797-afcaded7f8fe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" Jul 24 00:11:48.022580 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.022554 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8355e71-b509-4932-9797-afcaded7f8fe-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-hmvqn\" (UID: \"d8355e71-b509-4932-9797-afcaded7f8fe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" Jul 24 00:11:48.025047 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.025017 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8355e71-b509-4932-9797-afcaded7f8fe-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-hmvqn\" (UID: \"d8355e71-b509-4932-9797-afcaded7f8fe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" Jul 24 00:11:48.025134 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.025052 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d8355e71-b509-4932-9797-afcaded7f8fe-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-hmvqn\" (UID: \"d8355e71-b509-4932-9797-afcaded7f8fe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" Jul 24 00:11:48.036208 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.036188 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px9gw\" (UniqueName: \"kubernetes.io/projected/d8355e71-b509-4932-9797-afcaded7f8fe-kube-api-access-px9gw\") pod \"prometheus-operator-84d69f794d-hmvqn\" (UID: \"d8355e71-b509-4932-9797-afcaded7f8fe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" Jul 24 00:11:48.072199 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.072166 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" Jul 24 00:11:48.213480 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.213434 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-hmvqn"] Jul 24 00:11:48.216488 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.216462 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:48.218363 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:48.218324 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8355e71_b509_4932_9797_afcaded7f8fe.slice/crio-590cd825cdab9f7e76c822e6c063971ed3df4b8c86916118fca0b54fd7300b06 WatchSource:0}: Error finding container 590cd825cdab9f7e76c822e6c063971ed3df4b8c86916118fca0b54fd7300b06: Status 404 returned error can't find the container with id 590cd825cdab9f7e76c822e6c063971ed3df4b8c86916118fca0b54fd7300b06 Jul 24 00:11:48.287744 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.287630 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:48.421165 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.421082 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:48.519940 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.519857 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:48.586721 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.586686 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:48.587163 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.587136 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rsb8q" event={"ID":"0c87883b-9a82-4049-a788-b28dc9f49340","Type":"ContainerStarted","Data":"c392db7b55d4aeebfd0fcebb340925f3af1cefb8a78ce361513c1c92c8e41494"} Jul 24 00:11:48.590069 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.589992 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d7q6r" event={"ID":"1a539eef-502e-48e6-8020-2e6d9cc6ff5b","Type":"ContainerStarted","Data":"f10f89e1ca861a814b656a6b1ca5a918079eb9384068295ee4367674c911d5a3"} Jul 24 00:11:48.592030 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.591991 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-p88lj" event={"ID":"5ff612cc-1d5c-4195-b649-be35a283ce97","Type":"ContainerStarted","Data":"2fea798ab87de179e59e046fd4db4a8f4b5997b1ddb86159266f74c1e14a1e74"} Jul 24 00:11:48.593721 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.593695 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" event={"ID":"d8355e71-b509-4932-9797-afcaded7f8fe","Type":"ContainerStarted","Data":"590cd825cdab9f7e76c822e6c063971ed3df4b8c86916118fca0b54fd7300b06"} Jul 24 00:11:48.595554 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.595493 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fxm4z" event={"ID":"989c9067-460b-4838-b1b0-3da75e2a848f","Type":"ContainerStarted","Data":"161d056dc8953427948cab95735a0f0ecd2a4916b88751fdb32284e30d940a7a"} Jul 24 00:11:48.601269 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.600338 2560 generic.go:356] "Generic (PLEG): container finished" podID="e6bdd868-f7ad-4866-965a-d60339268678" containerID="ddd161b97193361832f3d9396d1679fc0833da0ea071c9fdbbb3413e4413bbfd" exitCode=0 Jul 24 00:11:48.601269 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:48.600377 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvwnr" event={"ID":"e6bdd868-f7ad-4866-965a-d60339268678","Type":"ContainerDied","Data":"ddd161b97193361832f3d9396d1679fc0833da0ea071c9fdbbb3413e4413bbfd"} Jul 24 00:11:49.611833 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:49.611789 2560 generic.go:356] "Generic (PLEG): container finished" podID="e6bdd868-f7ad-4866-965a-d60339268678" containerID="d678fac7ecc5687b4f0e8edb5c87934660898aa92ed49fcffddf38b9698f07ce" exitCode=0 Jul 24 00:11:49.612350 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:49.611875 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvwnr" event={"ID":"e6bdd868-f7ad-4866-965a-d60339268678","Type":"ContainerDied","Data":"d678fac7ecc5687b4f0e8edb5c87934660898aa92ed49fcffddf38b9698f07ce"} Jul 24 00:11:50.624586 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:50.624343 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvwnr" event={"ID":"e6bdd868-f7ad-4866-965a-d60339268678","Type":"ContainerStarted","Data":"aa4f0e9de30225d67b3e5250edea29a9100290722e89a698261d046c66a0cd2b"} Jul 24 00:11:50.652727 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:50.651866 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hvwnr" podStartSLOduration=5.338417126 podStartE2EDuration="37.651845126s" totalImagesPullingTime="32.313428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 00:11:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:11:50.650250178 +0000 UTC m=+39.255564497" watchObservedRunningTime="2026-07-24 00:11:50.651845126 +0000 UTC m=+39.257159445" Jul 24 00:11:50.775108 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:50.775025 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:50.831006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:50.830936 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:51.473525 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:51.473495 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:51.563725 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:51.563687 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:54.704949 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:54.704906 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:54.772223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:54.772184 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:55.573779 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.573739 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b895d5c5b-l9pv4"] Jul 24 00:11:55.615610 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.615576 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b895d5c5b-l9pv4"] Jul 24 00:11:55.615781 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.615729 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.618838 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.618679 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 24 00:11:55.618838 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.618682 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 24 00:11:55.619092 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.619074 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 24 00:11:55.619164 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.619145 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 24 00:11:55.621086 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.620886 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 24 00:11:55.621086 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.620932 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7nc75\"" Jul 24 00:11:55.791251 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.791210 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-console-config\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.791691 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.791262 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptlpw\" (UniqueName: \"kubernetes.io/projected/c572a5dd-23eb-496c-9835-3d565fe5631c-kube-api-access-ptlpw\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.791691 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.791329 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c572a5dd-23eb-496c-9835-3d565fe5631c-console-serving-cert\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.791691 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.791355 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c572a5dd-23eb-496c-9835-3d565fe5631c-console-oauth-config\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.791691 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.791471 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-oauth-serving-cert\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.791691 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.791511 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-service-ca\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.892587 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.892499 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-oauth-serving-cert\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.892587 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.892551 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-service-ca\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.892587 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.892582 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-console-config\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.892864 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.892610 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptlpw\" (UniqueName: \"kubernetes.io/projected/c572a5dd-23eb-496c-9835-3d565fe5631c-kube-api-access-ptlpw\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.892864 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.892690 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c572a5dd-23eb-496c-9835-3d565fe5631c-console-serving-cert\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.892864 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.892709 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c572a5dd-23eb-496c-9835-3d565fe5631c-console-oauth-config\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.893200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.893175 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-oauth-serving-cert\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.893727 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.893701 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-service-ca\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.894058 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.894015 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-console-config\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.897025 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.897002 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c572a5dd-23eb-496c-9835-3d565fe5631c-console-oauth-config\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.897133 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.897012 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c572a5dd-23eb-496c-9835-3d565fe5631c-console-serving-cert\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.901883 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.901688 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptlpw\" (UniqueName: \"kubernetes.io/projected/c572a5dd-23eb-496c-9835-3d565fe5631c-kube-api-access-ptlpw\") pod \"console-6b895d5c5b-l9pv4\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:55.927770 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:55.927740 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:11:56.434057 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:56.434028 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b895d5c5b-l9pv4"] Jul 24 00:11:56.437458 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:11:56.437429 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc572a5dd_23eb_496c_9835_3d565fe5631c.slice/crio-b83edc0c05d2bdfb493c37a2623392010b3362745d5dbd665f2de4aee1503966 WatchSource:0}: Error finding container b83edc0c05d2bdfb493c37a2623392010b3362745d5dbd665f2de4aee1503966: Status 404 returned error can't find the container with id b83edc0c05d2bdfb493c37a2623392010b3362745d5dbd665f2de4aee1503966 Jul 24 00:11:56.546809 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:56.546695 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:56.647280 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:56.647044 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:56.650090 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:56.649390 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rsb8q" event={"ID":"0c87883b-9a82-4049-a788-b28dc9f49340","Type":"ContainerStarted","Data":"d2f4f6d427f1159b26a4a2334763fd2b088a5345b2d98af21b14e7e6713db464"} Jul 24 00:11:56.657743 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:56.657662 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b895d5c5b-l9pv4" event={"ID":"c572a5dd-23eb-496c-9835-3d565fe5631c","Type":"ContainerStarted","Data":"b83edc0c05d2bdfb493c37a2623392010b3362745d5dbd665f2de4aee1503966"} Jul 24 00:11:56.662465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:56.662393 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" event={"ID":"d8355e71-b509-4932-9797-afcaded7f8fe","Type":"ContainerStarted","Data":"c77d211cd73318a95ce481b227288e6299a70966a7672811204fa921a61ef0c3"} Jul 24 00:11:56.664273 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:56.664215 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fxm4z" event={"ID":"989c9067-460b-4838-b1b0-3da75e2a848f","Type":"ContainerStarted","Data":"25c84bf77458a80b406387a1e102943e34a96819cf3047bf34e45b389052e59d"} Jul 24 00:11:56.671409 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:56.671360 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rsb8q" podStartSLOduration=3.144341174 podStartE2EDuration="11.671342696s" totalImagesPullingTime="8.527001522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:11:56.671057875 +0000 UTC m=+45.276372192" watchObservedRunningTime="2026-07-24 00:11:56.671342696 +0000 UTC m=+45.276657013" Jul 24 00:11:56.747575 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:56.747481 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:57.672932 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:57.672613 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" event={"ID":"d8355e71-b509-4932-9797-afcaded7f8fe","Type":"ContainerStarted","Data":"7d3bdbc3e8f5b41dcc7f4da7ec84525063384c6d0c78b4e0a3fbb1e75b0d9294"} Jul 24 00:11:57.677429 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:57.677353 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fxm4z" event={"ID":"989c9067-460b-4838-b1b0-3da75e2a848f","Type":"ContainerStarted","Data":"f12b20bf341419274c1911257b20e39e0dcd289a64d135300c28a6e5a6c3e22d"} Jul 24 00:11:57.677704 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:57.677633 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fxm4z" Jul 24 00:11:57.679789 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:57.679750 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d7q6r" event={"ID":"1a539eef-502e-48e6-8020-2e6d9cc6ff5b","Type":"ContainerStarted","Data":"e9a69be136f8f83046faa52de1dcebdbdec596e1d09226322d2e6ed63db42b8e"} Jul 24 00:11:57.714475 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:57.714408 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-hmvqn" podStartSLOduration=2.655930215 podStartE2EDuration="10.714390796s" totalImagesPullingTime="8.058460581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:11:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:11:57.69311575 +0000 UTC m=+46.298430068" watchObservedRunningTime="2026-07-24 00:11:57.714390796 +0000 UTC m=+46.319705103" Jul 24 00:11:57.715114 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:57.714944 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fxm4z" podStartSLOduration=4.187826076 podStartE2EDuration="12.714931164s" totalImagesPullingTime="8.527105088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:11:57.713829375 +0000 UTC m=+46.319143692" watchObservedRunningTime="2026-07-24 00:11:57.714931164 +0000 UTC m=+46.320245480" Jul 24 00:11:58.163498 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:58.163464 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:58.263848 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:58.263781 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:58.868424 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:58.868384 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:58.969203 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:58.969158 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:59.090018 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.089980 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:59.168053 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.167973 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn"] Jul 24 00:11:59.183412 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.183354 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn"] Jul 24 00:11:59.183581 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.183458 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" Jul 24 00:11:59.185981 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.185952 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 24 00:11:59.186379 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.186312 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-f5xrd\"" Jul 24 00:11:59.186491 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.186396 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 24 00:11:59.189883 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.189861 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt"] Jul 24 00:11:59.217458 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.217396 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:59.217736 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.217711 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt"] Jul 24 00:11:59.217887 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.217818 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.220482 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.220455 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 24 00:11:59.220791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.220735 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 24 00:11:59.220791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.220742 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 24 00:11:59.221017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.220996 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-blmlv\"" Jul 24 00:11:59.230055 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.229946 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vsn7g"] Jul 24 00:11:59.245824 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.245797 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.249530 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.249384 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pmp2f\"" Jul 24 00:11:59.249738 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.249531 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 00:11:59.249738 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.249583 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 00:11:59.249738 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.249628 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 00:11:59.307206 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.307170 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:59.325796 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.325760 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9d2b\" (UniqueName: \"kubernetes.io/projected/4acaa63f-fc09-4074-9806-f9e9a5b43142-kube-api-access-t9d2b\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.325974 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.325818 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4acaa63f-fc09-4074-9806-f9e9a5b43142-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.325974 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.325873 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4acaa63f-fc09-4074-9806-f9e9a5b43142-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.325974 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.325923 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da7c354c-8bc2-4e55-a070-d49f71cd4303-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-rdchn\" (UID: \"da7c354c-8bc2-4e55-a070-d49f71cd4303\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" Jul 24 00:11:59.325974 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.325960 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr6zd\" (UniqueName: \"kubernetes.io/projected/da7c354c-8bc2-4e55-a070-d49f71cd4303-kube-api-access-rr6zd\") pod \"openshift-state-metrics-b8f49f5c6-rdchn\" (UID: \"da7c354c-8bc2-4e55-a070-d49f71cd4303\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" Jul 24 00:11:59.326217 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.326104 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4acaa63f-fc09-4074-9806-f9e9a5b43142-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.326217 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.326168 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da7c354c-8bc2-4e55-a070-d49f71cd4303-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-rdchn\" (UID: \"da7c354c-8bc2-4e55-a070-d49f71cd4303\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" Jul 24 00:11:59.326323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.326217 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4acaa63f-fc09-4074-9806-f9e9a5b43142-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.326323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.326246 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/da7c354c-8bc2-4e55-a070-d49f71cd4303-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-rdchn\" (UID: \"da7c354c-8bc2-4e55-a070-d49f71cd4303\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" Jul 24 00:11:59.326323 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.326299 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4acaa63f-fc09-4074-9806-f9e9a5b43142-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.409164 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.409124 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:59.427177 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427089 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9d2b\" (UniqueName: \"kubernetes.io/projected/4acaa63f-fc09-4074-9806-f9e9a5b43142-kube-api-access-t9d2b\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.427177 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427130 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ecc5f68e-2d88-4dde-809b-c0654937b597-root\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.427177 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427169 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4acaa63f-fc09-4074-9806-f9e9a5b43142-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.427452 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427195 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ecc5f68e-2d88-4dde-809b-c0654937b597-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.427452 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427227 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4acaa63f-fc09-4074-9806-f9e9a5b43142-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.427452 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427252 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ecc5f68e-2d88-4dde-809b-c0654937b597-node-exporter-textfile\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.427452 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427295 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ecc5f68e-2d88-4dde-809b-c0654937b597-node-exporter-accelerators-collector-config\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.427673 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427474 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecc5f68e-2d88-4dde-809b-c0654937b597-metrics-client-ca\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.427673 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427513 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da7c354c-8bc2-4e55-a070-d49f71cd4303-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-rdchn\" (UID: \"da7c354c-8bc2-4e55-a070-d49f71cd4303\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" Jul 24 00:11:59.427673 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427555 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr6zd\" (UniqueName: \"kubernetes.io/projected/da7c354c-8bc2-4e55-a070-d49f71cd4303-kube-api-access-rr6zd\") pod \"openshift-state-metrics-b8f49f5c6-rdchn\" (UID: \"da7c354c-8bc2-4e55-a070-d49f71cd4303\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" Jul 24 00:11:59.427673 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427586 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ecc5f68e-2d88-4dde-809b-c0654937b597-node-exporter-tls\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.427673 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427615 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vkxz\" (UniqueName: \"kubernetes.io/projected/ecc5f68e-2d88-4dde-809b-c0654937b597-kube-api-access-8vkxz\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.427924 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427682 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4acaa63f-fc09-4074-9806-f9e9a5b43142-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.427924 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427728 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da7c354c-8bc2-4e55-a070-d49f71cd4303-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-rdchn\" (UID: \"da7c354c-8bc2-4e55-a070-d49f71cd4303\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" Jul 24 00:11:59.427924 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427756 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ecc5f68e-2d88-4dde-809b-c0654937b597-node-exporter-wtmp\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.427924 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427839 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4acaa63f-fc09-4074-9806-f9e9a5b43142-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.427924 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427874 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/da7c354c-8bc2-4e55-a070-d49f71cd4303-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-rdchn\" (UID: \"da7c354c-8bc2-4e55-a070-d49f71cd4303\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" Jul 24 00:11:59.427924 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427900 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4acaa63f-fc09-4074-9806-f9e9a5b43142-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.428218 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427938 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ecc5f68e-2d88-4dde-809b-c0654937b597-sys\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.428218 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.427957 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4acaa63f-fc09-4074-9806-f9e9a5b43142-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.428330 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.428270 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da7c354c-8bc2-4e55-a070-d49f71cd4303-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-rdchn\" (UID: \"da7c354c-8bc2-4e55-a070-d49f71cd4303\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" Jul 24 00:11:59.428412 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.428390 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4acaa63f-fc09-4074-9806-f9e9a5b43142-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.428586 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.428558 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4acaa63f-fc09-4074-9806-f9e9a5b43142-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.430868 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.430838 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4acaa63f-fc09-4074-9806-f9e9a5b43142-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.430985 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.430889 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/da7c354c-8bc2-4e55-a070-d49f71cd4303-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-rdchn\" (UID: \"da7c354c-8bc2-4e55-a070-d49f71cd4303\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" Jul 24 00:11:59.430985 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.430913 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/da7c354c-8bc2-4e55-a070-d49f71cd4303-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-rdchn\" (UID: \"da7c354c-8bc2-4e55-a070-d49f71cd4303\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" Jul 24 00:11:59.431559 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.431534 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4acaa63f-fc09-4074-9806-f9e9a5b43142-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.438325 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.438293 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr6zd\" (UniqueName: \"kubernetes.io/projected/da7c354c-8bc2-4e55-a070-d49f71cd4303-kube-api-access-rr6zd\") pod \"openshift-state-metrics-b8f49f5c6-rdchn\" (UID: \"da7c354c-8bc2-4e55-a070-d49f71cd4303\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" Jul 24 00:11:59.438900 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.438865 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9d2b\" (UniqueName: \"kubernetes.io/projected/4acaa63f-fc09-4074-9806-f9e9a5b43142-kube-api-access-t9d2b\") pod \"kube-state-metrics-56d9f944b4-nhxgt\" (UID: \"4acaa63f-fc09-4074-9806-f9e9a5b43142\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.498268 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.498239 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" Jul 24 00:11:59.508858 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.508818 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:59.528457 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.528427 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ecc5f68e-2d88-4dde-809b-c0654937b597-node-exporter-textfile\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.528620 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.528469 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ecc5f68e-2d88-4dde-809b-c0654937b597-node-exporter-accelerators-collector-config\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.528620 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.528496 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecc5f68e-2d88-4dde-809b-c0654937b597-metrics-client-ca\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.528620 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.528537 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ecc5f68e-2d88-4dde-809b-c0654937b597-node-exporter-tls\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.528620 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.528567 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vkxz\" (UniqueName: \"kubernetes.io/projected/ecc5f68e-2d88-4dde-809b-c0654937b597-kube-api-access-8vkxz\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.528871 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.528623 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ecc5f68e-2d88-4dde-809b-c0654937b597-node-exporter-wtmp\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.528871 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.528703 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ecc5f68e-2d88-4dde-809b-c0654937b597-sys\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.528871 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.528728 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ecc5f68e-2d88-4dde-809b-c0654937b597-root\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.528871 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.528773 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ecc5f68e-2d88-4dde-809b-c0654937b597-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.528871 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.528833 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ecc5f68e-2d88-4dde-809b-c0654937b597-node-exporter-textfile\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.528871 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.528866 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ecc5f68e-2d88-4dde-809b-c0654937b597-sys\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.529157 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.528883 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ecc5f68e-2d88-4dde-809b-c0654937b597-node-exporter-wtmp\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.529157 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.528903 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ecc5f68e-2d88-4dde-809b-c0654937b597-root\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.529157 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.529036 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ecc5f68e-2d88-4dde-809b-c0654937b597-node-exporter-accelerators-collector-config\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.529314 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.529296 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecc5f68e-2d88-4dde-809b-c0654937b597-metrics-client-ca\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.529481 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.529401 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" Jul 24 00:11:59.531381 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.531361 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ecc5f68e-2d88-4dde-809b-c0654937b597-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.531532 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.531507 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ecc5f68e-2d88-4dde-809b-c0654937b597-node-exporter-tls\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.537450 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.537425 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vkxz\" (UniqueName: \"kubernetes.io/projected/ecc5f68e-2d88-4dde-809b-c0654937b597-kube-api-access-8vkxz\") pod \"node-exporter-vsn7g\" (UID: \"ecc5f68e-2d88-4dde-809b-c0654937b597\") " pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.559534 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.559496 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vsn7g" Jul 24 00:11:59.609105 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.609073 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:11:59.899050 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:11:59.899018 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:00.203137 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.203056 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 00:12:00.222292 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.222257 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.224736 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.224708 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 24 00:12:00.226060 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.226039 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 24 00:12:00.226453 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.226280 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 24 00:12:00.226453 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.226291 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 24 00:12:00.226453 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.226290 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-j2cs9\"" Jul 24 00:12:00.226453 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.226388 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 24 00:12:00.226453 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.226312 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 24 00:12:00.226791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.226584 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 24 00:12:00.226791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.226612 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 24 00:12:00.226994 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.226975 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 24 00:12:00.232416 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.232326 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 00:12:00.239552 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.239528 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:00.303052 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.303020 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:00.335264 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.335228 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.335455 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.335298 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.335455 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.335325 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-config-out\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.335455 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.335343 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.335455 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.335366 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.335689 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.335488 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.335689 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.335529 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.335689 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.335566 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-config-volume\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.335689 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.335604 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-web-config\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.335689 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.335664 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.335908 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.335708 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.335908 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.335761 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.335908 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.335786 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrrvd\" (UniqueName: \"kubernetes.io/projected/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-kube-api-access-hrrvd\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.437099 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.437061 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.437277 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.437127 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.437277 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.437159 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-config-out\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.437277 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.437186 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.437441 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.437316 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.437441 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.437384 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.437441 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.437412 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.437674 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.437551 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-config-volume\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.437674 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.437577 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-web-config\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.437674 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.437599 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.437674 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.437632 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.437945 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.437875 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.437945 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.437909 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrrvd\" (UniqueName: \"kubernetes.io/projected/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-kube-api-access-hrrvd\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.437945 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.437911 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.439511 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.438282 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.439511 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:00.438402 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-alertmanager-trusted-ca-bundle podName:8880ee1f-1a6e-4a69-82c2-b3e9be9ae565 nodeName:}" failed. No retries permitted until 2026-07-24 00:12:00.938384094 +0000 UTC m=+49.543698409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565") : configmap references non-existent config key: ca-bundle.crt Jul 24 00:12:00.440916 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.440889 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.441039 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.440930 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-config-volume\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.441213 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.441166 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.441722 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.441677 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.442420 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.441984 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-web-config\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.442512 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.442459 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.442959 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.442858 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-config-out\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.442959 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.442868 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.443980 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.443959 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.446869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.446847 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrrvd\" (UniqueName: \"kubernetes.io/projected/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-kube-api-access-hrrvd\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.476298 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.476231 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:00.578462 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.578421 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:00.678762 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.678723 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:00.942959 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.942922 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:00.944049 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:00.944022 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:01.133627 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:01.133589 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:01.148200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:01.148166 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:01.797695 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:01.797659 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:01.930930 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:01.930897 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:02.026156 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.026113 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:02.110976 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.110894 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:02.119253 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.119221 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2"] Jul 24 00:12:02.131000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.130973 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.133903 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.133880 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 24 00:12:02.134561 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.134397 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 24 00:12:02.134704 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.134665 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 24 00:12:02.134773 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.134748 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-czv25\"" Jul 24 00:12:02.135966 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.135817 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-87u23j4vcrtma\"" Jul 24 00:12:02.135966 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.135944 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 24 00:12:02.137560 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.137477 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2"] Jul 24 00:12:02.137985 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.137964 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 24 00:12:02.153659 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.153614 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6819273d-e59d-4321-b9f3-e3afd59cf648-metrics-client-ca\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.153801 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.153681 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-grpc-tls\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.153879 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.153825 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.153938 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.153873 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.153938 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.153915 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-thanos-querier-tls\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.154043 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.153966 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.154043 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.154036 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmtc2\" (UniqueName: \"kubernetes.io/projected/6819273d-e59d-4321-b9f3-e3afd59cf648-kube-api-access-jmtc2\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.154150 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.154084 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.254863 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.254826 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-thanos-querier-tls\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.255087 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.254890 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.255087 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.254954 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmtc2\" (UniqueName: \"kubernetes.io/projected/6819273d-e59d-4321-b9f3-e3afd59cf648-kube-api-access-jmtc2\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.255087 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.254998 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.255087 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.255034 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6819273d-e59d-4321-b9f3-e3afd59cf648-metrics-client-ca\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.255087 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.255067 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-grpc-tls\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.255352 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.255136 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.255352 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.255162 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.256322 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.256292 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6819273d-e59d-4321-b9f3-e3afd59cf648-metrics-client-ca\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.257936 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.257886 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.258481 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.258457 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.258597 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.258457 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-grpc-tls\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.258597 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.258508 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-thanos-querier-tls\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.258928 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.258907 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.259055 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.259026 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6819273d-e59d-4321-b9f3-e3afd59cf648-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.264820 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.264799 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmtc2\" (UniqueName: \"kubernetes.io/projected/6819273d-e59d-4321-b9f3-e3afd59cf648-kube-api-access-jmtc2\") pod \"thanos-querier-6bcd874fcc-r2sb2\" (UID: \"6819273d-e59d-4321-b9f3-e3afd59cf648\") " pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.432804 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.432715 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:02.443628 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.443597 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:02.552278 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:02.552233 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:03.472073 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.472038 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:03.558447 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.558407 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6d8f567488-7cnck"] Jul 24 00:12:03.577138 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.577105 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:03.583632 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.583606 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d8f567488-7cnck"] Jul 24 00:12:03.583807 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.583721 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.587526 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.587486 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 24 00:12:03.587718 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.587691 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 24 00:12:03.588712 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.588690 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-529tl\"" Jul 24 00:12:03.588932 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.588906 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 24 00:12:03.589049 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.588960 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 24 00:12:03.589049 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.588972 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-187qbkr195dj2\"" Jul 24 00:12:03.665553 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.665513 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b164d179-d56a-451e-9b4e-00b001172f51-client-ca-bundle\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.665744 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.665568 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nf6x\" (UniqueName: \"kubernetes.io/projected/b164d179-d56a-451e-9b4e-00b001172f51-kube-api-access-5nf6x\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.665744 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.665621 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b164d179-d56a-451e-9b4e-00b001172f51-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.665744 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.665682 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b164d179-d56a-451e-9b4e-00b001172f51-metrics-server-audit-profiles\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.665880 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.665746 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b164d179-d56a-451e-9b4e-00b001172f51-audit-log\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.665880 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.665798 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b164d179-d56a-451e-9b4e-00b001172f51-secret-metrics-server-client-certs\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.665880 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.665858 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b164d179-d56a-451e-9b4e-00b001172f51-secret-metrics-server-tls\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.766265 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.766228 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b164d179-d56a-451e-9b4e-00b001172f51-client-ca-bundle\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.766465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.766280 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nf6x\" (UniqueName: \"kubernetes.io/projected/b164d179-d56a-451e-9b4e-00b001172f51-kube-api-access-5nf6x\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.766465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.766306 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b164d179-d56a-451e-9b4e-00b001172f51-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.766465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.766331 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b164d179-d56a-451e-9b4e-00b001172f51-metrics-server-audit-profiles\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.766465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.766391 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b164d179-d56a-451e-9b4e-00b001172f51-audit-log\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.766465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.766436 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b164d179-d56a-451e-9b4e-00b001172f51-secret-metrics-server-client-certs\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.766705 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.766470 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b164d179-d56a-451e-9b4e-00b001172f51-secret-metrics-server-tls\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.767170 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.767101 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b164d179-d56a-451e-9b4e-00b001172f51-audit-log\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.767297 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.767175 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b164d179-d56a-451e-9b4e-00b001172f51-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.767661 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.767582 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b164d179-d56a-451e-9b4e-00b001172f51-metrics-server-audit-profiles\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.769383 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.769354 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b164d179-d56a-451e-9b4e-00b001172f51-secret-metrics-server-client-certs\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.769491 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.769416 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b164d179-d56a-451e-9b4e-00b001172f51-client-ca-bundle\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.769491 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.769459 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b164d179-d56a-451e-9b4e-00b001172f51-secret-metrics-server-tls\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.775515 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.775458 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nf6x\" (UniqueName: \"kubernetes.io/projected/b164d179-d56a-451e-9b4e-00b001172f51-kube-api-access-5nf6x\") pod \"metrics-server-6d8f567488-7cnck\" (UID: \"b164d179-d56a-451e-9b4e-00b001172f51\") " pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.869081 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.869037 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk"] Jul 24 00:12:03.891579 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.891542 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk"] Jul 24 00:12:03.891768 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.891698 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk" Jul 24 00:12:03.894143 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.894114 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:03.894766 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.894568 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 24 00:12:03.894766 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.894717 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5dctg\"" Jul 24 00:12:03.967539 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.967501 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2pml\" (UniqueName: \"kubernetes.io/projected/1cf6076a-b8f1-44aa-a565-28328b56a8d9-kube-api-access-w2pml\") pod \"monitoring-plugin-68b7fc9574-rvvkk\" (UID: \"1cf6076a-b8f1-44aa-a565-28328b56a8d9\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk" Jul 24 00:12:03.967734 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:03.967584 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1cf6076a-b8f1-44aa-a565-28328b56a8d9-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-rvvkk\" (UID: \"1cf6076a-b8f1-44aa-a565-28328b56a8d9\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk" Jul 24 00:12:04.068258 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.068180 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1cf6076a-b8f1-44aa-a565-28328b56a8d9-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-rvvkk\" (UID: \"1cf6076a-b8f1-44aa-a565-28328b56a8d9\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk" Jul 24 00:12:04.068418 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.068313 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2pml\" (UniqueName: \"kubernetes.io/projected/1cf6076a-b8f1-44aa-a565-28328b56a8d9-kube-api-access-w2pml\") pod \"monitoring-plugin-68b7fc9574-rvvkk\" (UID: \"1cf6076a-b8f1-44aa-a565-28328b56a8d9\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk" Jul 24 00:12:04.071069 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.071044 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1cf6076a-b8f1-44aa-a565-28328b56a8d9-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-rvvkk\" (UID: \"1cf6076a-b8f1-44aa-a565-28328b56a8d9\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk" Jul 24 00:12:04.082002 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.081973 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2pml\" (UniqueName: \"kubernetes.io/projected/1cf6076a-b8f1-44aa-a565-28328b56a8d9-kube-api-access-w2pml\") pod \"monitoring-plugin-68b7fc9574-rvvkk\" (UID: \"1cf6076a-b8f1-44aa-a565-28328b56a8d9\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk" Jul 24 00:12:04.140016 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.139979 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-846dc5d995-bhk8z"] Jul 24 00:12:04.160058 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.160026 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-846dc5d995-bhk8z"] Jul 24 00:12:04.160233 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.160171 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.169484 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.169455 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-console-config\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.169659 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.169497 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-trusted-ca-bundle\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.169659 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.169545 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0390497f-31b8-4148-a8db-5436b4d7007e-console-serving-cert\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.169659 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.169620 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-oauth-serving-cert\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.169830 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.169679 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv8x4\" (UniqueName: \"kubernetes.io/projected/0390497f-31b8-4148-a8db-5436b4d7007e-kube-api-access-fv8x4\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.169882 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.169806 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0390497f-31b8-4148-a8db-5436b4d7007e-console-oauth-config\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.169936 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.169883 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-service-ca\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.170886 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.170851 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 24 00:12:04.191845 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.191803 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:04.202726 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.202696 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk" Jul 24 00:12:04.255232 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.255185 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:04.270978 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.270941 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-console-config\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.270978 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.270986 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-trusted-ca-bundle\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.271223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.271023 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0390497f-31b8-4148-a8db-5436b4d7007e-console-serving-cert\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.271223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.271039 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-oauth-serving-cert\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.271223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.271059 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv8x4\" (UniqueName: \"kubernetes.io/projected/0390497f-31b8-4148-a8db-5436b4d7007e-kube-api-access-fv8x4\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.271223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.271126 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0390497f-31b8-4148-a8db-5436b4d7007e-console-oauth-config\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.271223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.271147 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-service-ca\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.271822 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.271795 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-console-config\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.271928 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.271856 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-oauth-serving-cert\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.272349 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.272324 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-service-ca\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.272452 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.272335 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-trusted-ca-bundle\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.274093 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.274067 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0390497f-31b8-4148-a8db-5436b4d7007e-console-oauth-config\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.274229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.274179 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0390497f-31b8-4148-a8db-5436b4d7007e-console-serving-cert\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.280408 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.280381 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv8x4\" (UniqueName: \"kubernetes.io/projected/0390497f-31b8-4148-a8db-5436b4d7007e-kube-api-access-fv8x4\") pod \"console-846dc5d995-bhk8z\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:04.356020 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.355932 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:04.421328 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.421285 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:04.472694 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:04.472623 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:05.369871 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.369824 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 00:12:05.392373 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.392337 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:05.395031 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.394952 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 00:12:05.395031 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.394976 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.397898 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.397484 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 24 00:12:05.397898 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.397756 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jbx9f\"" Jul 24 00:12:05.400762 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.400736 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 24 00:12:05.400762 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.400756 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 24 00:12:05.400942 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.400805 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 24 00:12:05.400942 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.400817 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 24 00:12:05.400942 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.400761 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f49kd2mto0hi\"" Jul 24 00:12:05.401106 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.401084 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 24 00:12:05.401176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.401130 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 24 00:12:05.401250 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.401225 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 24 00:12:05.401466 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.401242 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 24 00:12:05.401466 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.401366 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 24 00:12:05.401895 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.401874 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 24 00:12:05.402054 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.401974 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 24 00:12:05.403989 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.403968 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 24 00:12:05.407600 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.407523 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 24 00:12:05.481777 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.481741 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.481787 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.481817 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw9sm\" (UniqueName: \"kubernetes.io/projected/2f0a1517-451f-41a0-91b8-382111995fff-kube-api-access-jw9sm\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.481872 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.481929 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.481962 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.482069 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.482135 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.482181 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-config\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482229 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.482217 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482701 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.482263 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482701 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.482296 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482701 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.482327 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482701 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.482351 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482701 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.482407 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f0a1517-451f-41a0-91b8-382111995fff-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482701 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.482438 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482701 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.482467 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482701 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.482502 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f0a1517-451f-41a0-91b8-382111995fff-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482701 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.482530 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.482701 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.482575 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.584395 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584344 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.584578 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584422 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-config\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.584578 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584459 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.584578 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584490 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.584578 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584521 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.584578 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584549 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.584578 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584572 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.584886 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584622 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f0a1517-451f-41a0-91b8-382111995fff-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.584886 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584661 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.584886 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584683 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.584886 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584714 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f0a1517-451f-41a0-91b8-382111995fff-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.584886 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584743 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.584886 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584837 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.585180 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584897 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.585180 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584927 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.585180 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584953 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw9sm\" (UniqueName: \"kubernetes.io/projected/2f0a1517-451f-41a0-91b8-382111995fff-kube-api-access-jw9sm\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.585180 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.584984 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.585180 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.585014 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.585180 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.585052 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.585180 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.585108 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.585529 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.585503 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.585625 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.585600 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.586327 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.586039 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.586753 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.586587 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.586927 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.586871 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.590007 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.589181 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.590007 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.589601 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.590007 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.589934 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.590007 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.588630 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.590281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.590216 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.590281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.590240 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.590418 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.590398 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-config\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.590657 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.590604 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.591101 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.591080 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f0a1517-451f-41a0-91b8-382111995fff-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.591191 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.591146 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f0a1517-451f-41a0-91b8-382111995fff-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.591265 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.591243 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.591573 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.591549 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.594178 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.593339 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.594888 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.594859 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.595383 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.595358 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw9sm\" (UniqueName: \"kubernetes.io/projected/2f0a1517-451f-41a0-91b8-382111995fff-kube-api-access-jw9sm\") pod \"prometheus-k8s-0\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:05.709072 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:05.708995 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:06.327870 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:06.327823 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:06.451873 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:06.451836 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:07.685696 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:07.685662 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fxm4z" Jul 24 00:12:09.689544 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:09.689515 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:09.792421 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:09.792381 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:09.797707 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:12:09.797676 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecc5f68e_2d88_4dde_809b_c0654937b597.slice/crio-2771f972f294cdd6d2ca62391f511a78aa953637c2505253e262aa92fa098fc1 WatchSource:0}: Error finding container 2771f972f294cdd6d2ca62391f511a78aa953637c2505253e262aa92fa098fc1: Status 404 returned error can't find the container with id 2771f972f294cdd6d2ca62391f511a78aa953637c2505253e262aa92fa098fc1 Jul 24 00:12:09.953172 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:09.953144 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn"] Jul 24 00:12:10.014672 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:12:10.014621 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda7c354c_8bc2_4e55_a070_d49f71cd4303.slice/crio-dc2b085eb912c9c9fc2a2a1a92b44deb846022fdac2019dfc29c01777cf5ea7b WatchSource:0}: Error finding container dc2b085eb912c9c9fc2a2a1a92b44deb846022fdac2019dfc29c01777cf5ea7b: Status 404 returned error can't find the container with id dc2b085eb912c9c9fc2a2a1a92b44deb846022fdac2019dfc29c01777cf5ea7b Jul 24 00:12:10.210796 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.210389 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d8f567488-7cnck"] Jul 24 00:12:10.218333 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.217709 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk"] Jul 24 00:12:10.247747 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.247563 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-846dc5d995-bhk8z"] Jul 24 00:12:10.266552 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.266393 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt"] Jul 24 00:12:10.286181 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.286066 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2"] Jul 24 00:12:10.294386 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.294361 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 00:12:10.296567 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.296542 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 00:12:10.355982 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.355924 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:10.363559 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:12:10.363533 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb164d179_d56a_451e_9b4e_00b001172f51.slice/crio-85c5632fe24254c748e277a281a56c783f85897c8545155203b0768fa1eca55c WatchSource:0}: Error finding container 85c5632fe24254c748e277a281a56c783f85897c8545155203b0768fa1eca55c: Status 404 returned error can't find the container with id 85c5632fe24254c748e277a281a56c783f85897c8545155203b0768fa1eca55c Jul 24 00:12:10.364902 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:12:10.364868 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cf6076a_b8f1_44aa_a565_28328b56a8d9.slice/crio-b8a139cb12d82e8774c81d4c6767b38ef4152b4b658dfc5e0f8d3225a1a83955 WatchSource:0}: Error finding container b8a139cb12d82e8774c81d4c6767b38ef4152b4b658dfc5e0f8d3225a1a83955: Status 404 returned error can't find the container with id b8a139cb12d82e8774c81d4c6767b38ef4152b4b658dfc5e0f8d3225a1a83955 Jul 24 00:12:10.366005 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:12:10.365593 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0390497f_31b8_4148_a8db_5436b4d7007e.slice/crio-ffdd1d95ca06d394cec89a22ad9f8a1fff49a0c16056514ae1262df38a9dd5f7 WatchSource:0}: Error finding container ffdd1d95ca06d394cec89a22ad9f8a1fff49a0c16056514ae1262df38a9dd5f7: Status 404 returned error can't find the container with id ffdd1d95ca06d394cec89a22ad9f8a1fff49a0c16056514ae1262df38a9dd5f7 Jul 24 00:12:10.367237 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:12:10.367097 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4acaa63f_fc09_4074_9806_f9e9a5b43142.slice/crio-a715a6c2db13723b51ccb732e2e4e29a4770b45843c80ee93c6e23792f0a3360 WatchSource:0}: Error finding container a715a6c2db13723b51ccb732e2e4e29a4770b45843c80ee93c6e23792f0a3360: Status 404 returned error can't find the container with id a715a6c2db13723b51ccb732e2e4e29a4770b45843c80ee93c6e23792f0a3360 Jul 24 00:12:10.368243 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:12:10.368014 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6819273d_e59d_4321_b9f3_e3afd59cf648.slice/crio-c182ec62a29b3cb3c5f7faa454b75bab06db72c6ff5cf5a1324d7482be8882f2 WatchSource:0}: Error finding container c182ec62a29b3cb3c5f7faa454b75bab06db72c6ff5cf5a1324d7482be8882f2: Status 404 returned error can't find the container with id c182ec62a29b3cb3c5f7faa454b75bab06db72c6ff5cf5a1324d7482be8882f2 Jul 24 00:12:10.371413 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:12:10.371332 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8880ee1f_1a6e_4a69_82c2_b3e9be9ae565.slice/crio-5338279c690ea0abad0ec161e5557d4e297db9a96133aacc6ffb698a55a0f791 WatchSource:0}: Error finding container 5338279c690ea0abad0ec161e5557d4e297db9a96133aacc6ffb698a55a0f791: Status 404 returned error can't find the container with id 5338279c690ea0abad0ec161e5557d4e297db9a96133aacc6ffb698a55a0f791 Jul 24 00:12:10.455626 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.455107 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:10.731036 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.730886 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f0a1517-451f-41a0-91b8-382111995fff","Type":"ContainerStarted","Data":"5afb060c7548ba2b14005124ac754dda40919de7bbcf5452bc23d749151b6a15"} Jul 24 00:12:10.733281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.733151 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846dc5d995-bhk8z" event={"ID":"0390497f-31b8-4148-a8db-5436b4d7007e","Type":"ContainerStarted","Data":"ddbcb8daff9506b8dc976bb3e3322b8ea3edefd091e15db615ca90d58e3b2943"} Jul 24 00:12:10.733281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.733205 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846dc5d995-bhk8z" event={"ID":"0390497f-31b8-4148-a8db-5436b4d7007e","Type":"ContainerStarted","Data":"ffdd1d95ca06d394cec89a22ad9f8a1fff49a0c16056514ae1262df38a9dd5f7"} Jul 24 00:12:10.735061 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.735026 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vsn7g" event={"ID":"ecc5f68e-2d88-4dde-809b-c0654937b597","Type":"ContainerStarted","Data":"2771f972f294cdd6d2ca62391f511a78aa953637c2505253e262aa92fa098fc1"} Jul 24 00:12:10.738120 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.738032 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d7q6r" event={"ID":"1a539eef-502e-48e6-8020-2e6d9cc6ff5b","Type":"ContainerStarted","Data":"5c961b312563db5eaa66cd1f84608a26ab2247497ae4f5422a309b8937dad659"} Jul 24 00:12:10.740259 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.740232 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-p88lj" event={"ID":"5ff612cc-1d5c-4195-b649-be35a283ce97","Type":"ContainerStarted","Data":"5e973017f08d23649f2e66de179f81fdf8c337a5235b94545dd7cce2a71e50d2"} Jul 24 00:12:10.740671 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.740622 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-p88lj" Jul 24 00:12:10.742914 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.742889 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b895d5c5b-l9pv4" event={"ID":"c572a5dd-23eb-496c-9835-3d565fe5631c","Type":"ContainerStarted","Data":"6b1b9435a7960d775c7ba8a565f8e4381330ea8ac2bf91a6251aad84163b0b7a"} Jul 24 00:12:10.744253 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.744226 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk" event={"ID":"1cf6076a-b8f1-44aa-a565-28328b56a8d9","Type":"ContainerStarted","Data":"b8a139cb12d82e8774c81d4c6767b38ef4152b4b658dfc5e0f8d3225a1a83955"} Jul 24 00:12:10.745689 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.745669 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" event={"ID":"b164d179-d56a-451e-9b4e-00b001172f51","Type":"ContainerStarted","Data":"85c5632fe24254c748e277a281a56c783f85897c8545155203b0768fa1eca55c"} Jul 24 00:12:10.747145 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.747122 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565","Type":"ContainerStarted","Data":"5338279c690ea0abad0ec161e5557d4e297db9a96133aacc6ffb698a55a0f791"} Jul 24 00:12:10.748559 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.748535 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" event={"ID":"6819273d-e59d-4321-b9f3-e3afd59cf648","Type":"ContainerStarted","Data":"c182ec62a29b3cb3c5f7faa454b75bab06db72c6ff5cf5a1324d7482be8882f2"} Jul 24 00:12:10.751056 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.751035 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" event={"ID":"4acaa63f-fc09-4074-9806-f9e9a5b43142","Type":"ContainerStarted","Data":"a715a6c2db13723b51ccb732e2e4e29a4770b45843c80ee93c6e23792f0a3360"} Jul 24 00:12:10.752593 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.751388 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-846dc5d995-bhk8z" podStartSLOduration=6.75137356 podStartE2EDuration="6.75137356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:12:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:10.750749843 +0000 UTC m=+59.356064160" watchObservedRunningTime="2026-07-24 00:12:10.75137356 +0000 UTC m=+59.356687879" Jul 24 00:12:10.754466 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.754372 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" event={"ID":"da7c354c-8bc2-4e55-a070-d49f71cd4303","Type":"ContainerStarted","Data":"2145e9e7f3ad286c960e79f13138d64e9617673f3a1dd52c5591d861bf3196f1"} Jul 24 00:12:10.754466 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.754446 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-p88lj" Jul 24 00:12:10.754466 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.754462 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" event={"ID":"da7c354c-8bc2-4e55-a070-d49f71cd4303","Type":"ContainerStarted","Data":"85fd841091c56020abae8b72dd55aadb928fc364b76073cd8aca0b202f9f001c"} Jul 24 00:12:10.755017 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.754492 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" event={"ID":"da7c354c-8bc2-4e55-a070-d49f71cd4303","Type":"ContainerStarted","Data":"dc2b085eb912c9c9fc2a2a1a92b44deb846022fdac2019dfc29c01777cf5ea7b"} Jul 24 00:12:10.768019 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.767953 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-p88lj" podStartSLOduration=3.722746973 podStartE2EDuration="25.767941432s" totalImagesPullingTime="22.045194459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:10.766758309 +0000 UTC m=+59.372072626" watchObservedRunningTime="2026-07-24 00:12:10.767941432 +0000 UTC m=+59.373255747" Jul 24 00:12:10.783512 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:10.783460 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b895d5c5b-l9pv4" podStartSLOduration=2.42582594 podStartE2EDuration="15.783439841s" totalImagesPullingTime="13.357613901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:10.782500268 +0000 UTC m=+59.387814617" watchObservedRunningTime="2026-07-24 00:12:10.783439841 +0000 UTC m=+59.388754157" Jul 24 00:12:11.776548 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:11.774897 2560 generic.go:356] "Generic (PLEG): container finished" podID="ecc5f68e-2d88-4dde-809b-c0654937b597" containerID="c870f0f7e20cb6226328ab8d06e3e6f7cff80bfe9cb749a67920bd485474e418" exitCode=0 Jul 24 00:12:11.776548 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:11.776150 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vsn7g" event={"ID":"ecc5f68e-2d88-4dde-809b-c0654937b597","Type":"ContainerDied","Data":"c870f0f7e20cb6226328ab8d06e3e6f7cff80bfe9cb749a67920bd485474e418"} Jul 24 00:12:12.631713 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:12.631635 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:12.723404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:12.723301 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:12.809816 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:12.809699 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vsn7g" event={"ID":"ecc5f68e-2d88-4dde-809b-c0654937b597","Type":"ContainerStarted","Data":"bcdbe2d0a775fccc5e8cc62d08bcd69b04a822c1d669932840730a6a6722a421"} Jul 24 00:12:12.809816 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:12.809756 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vsn7g" event={"ID":"ecc5f68e-2d88-4dde-809b-c0654937b597","Type":"ContainerStarted","Data":"46e253f76c2d9de8d3676c18b2fc3562b20ff9f5b80818cb44b2e82cdb3e31ae"} Jul 24 00:12:12.830958 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:12.829753 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vsn7g" podStartSLOduration=12.113227878 podStartE2EDuration="13.829733783s" totalImagesPullingTime="1.550829018s" totalInitContainerRuntime="165.676887ms" isStatefulPod=true podCreationTimestamp="2026-07-24 00:11:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:12.829518012 +0000 UTC m=+61.434832328" watchObservedRunningTime="2026-07-24 00:12:12.829733783 +0000 UTC m=+61.435048082" Jul 24 00:12:13.371782 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:13.371739 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:13.594369 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:13.594226 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rfgrr" Jul 24 00:12:14.026930 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:14.026865 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:14.463035 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:14.462820 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:14.473813 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:14.473785 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:14.474086 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:14.473842 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:14.480294 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:14.480266 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:14.825523 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:14.825487 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:12:14.882860 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:14.882724 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b895d5c5b-l9pv4"] Jul 24 00:12:15.181437 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:15.181360 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:15.279062 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:15.279023 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:15.928712 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:15.928682 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:12:16.168403 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.168352 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:16.269021 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.268985 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:16.616704 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.616590 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs\") pod \"network-metrics-daemon-lqb6d\" (UID: \"3266eb6f-f15e-4b80-ba25-b0f9807214e6\") " pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:12:16.616926 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.616757 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:12:16.619975 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.619750 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 00:12:16.620795 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.620767 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 00:12:16.630183 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.630159 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff0d6407-0175-4c0e-a54c-528dae80387a-original-pull-secret\") pod \"global-pull-secret-syncer-k66hc\" (UID: \"ff0d6407-0175-4c0e-a54c-528dae80387a\") " pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:12:16.631231 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.631198 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3266eb6f-f15e-4b80-ba25-b0f9807214e6-metrics-certs\") pod \"network-metrics-daemon-lqb6d\" (UID: \"3266eb6f-f15e-4b80-ba25-b0f9807214e6\") " pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:12:16.649517 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.649482 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:16.717597 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.717548 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cthx\" (UniqueName: \"kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx\") pod \"network-check-target-2rr2t\" (UID: \"0b1308db-61d6-49a8-b546-cf5bff91afb5\") " pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:12:16.720568 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.720537 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 00:12:16.730701 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.730670 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 00:12:16.741268 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.741205 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cthx\" (UniqueName: \"kubernetes.io/projected/0b1308db-61d6-49a8-b546-cf5bff91afb5-kube-api-access-4cthx\") pod \"network-check-target-2rr2t\" (UID: \"0b1308db-61d6-49a8-b546-cf5bff91afb5\") " pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:12:16.753867 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.753676 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rvqg9\"" Jul 24 00:12:16.754485 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.754454 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:16.755852 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.755830 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k66hc" Jul 24 00:12:16.761286 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.761227 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqb6d" Jul 24 00:12:16.853186 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.853146 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:16.915959 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.915817 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:16.955324 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.955289 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-79cjj\"" Jul 24 00:12:16.963693 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:16.963663 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:12:19.830629 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:19.830571 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2rr2t"] Jul 24 00:12:19.845117 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:19.845096 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k66hc"] Jul 24 00:12:19.864327 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:19.864297 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lqb6d"] Jul 24 00:12:19.891847 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:12:19.891818 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b1308db_61d6_49a8_b546_cf5bff91afb5.slice/crio-026da9d209addd5850ca92dcada262aede05780e49cf6dd5742a275491e5352b WatchSource:0}: Error finding container 026da9d209addd5850ca92dcada262aede05780e49cf6dd5742a275491e5352b: Status 404 returned error can't find the container with id 026da9d209addd5850ca92dcada262aede05780e49cf6dd5742a275491e5352b Jul 24 00:12:19.892489 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:12:19.892468 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff0d6407_0175_4c0e_a54c_528dae80387a.slice/crio-19441127e1979f125118fcfda174fd2acdbb5f25b0641140e8fc216a1f697bf0 WatchSource:0}: Error finding container 19441127e1979f125118fcfda174fd2acdbb5f25b0641140e8fc216a1f697bf0: Status 404 returned error can't find the container with id 19441127e1979f125118fcfda174fd2acdbb5f25b0641140e8fc216a1f697bf0 Jul 24 00:12:19.893764 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:12:19.893735 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3266eb6f_f15e_4b80_ba25_b0f9807214e6.slice/crio-b3927e7e0dbf5ba1a3ab058c7d27fd94202ef2bab7694169c722bb949ed4c679 WatchSource:0}: Error finding container b3927e7e0dbf5ba1a3ab058c7d27fd94202ef2bab7694169c722bb949ed4c679: Status 404 returned error can't find the container with id b3927e7e0dbf5ba1a3ab058c7d27fd94202ef2bab7694169c722bb949ed4c679 Jul 24 00:12:20.855908 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.855794 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" event={"ID":"da7c354c-8bc2-4e55-a070-d49f71cd4303","Type":"ContainerStarted","Data":"c155d6f11270d7d62a4c4ea6b048809e19c2e28aa7b2c43195a449258dc84f76"} Jul 24 00:12:20.857938 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.857860 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k66hc" event={"ID":"ff0d6407-0175-4c0e-a54c-528dae80387a","Type":"ContainerStarted","Data":"19441127e1979f125118fcfda174fd2acdbb5f25b0641140e8fc216a1f697bf0"} Jul 24 00:12:20.861506 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.861473 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lqb6d" event={"ID":"3266eb6f-f15e-4b80-ba25-b0f9807214e6","Type":"ContainerStarted","Data":"b3927e7e0dbf5ba1a3ab058c7d27fd94202ef2bab7694169c722bb949ed4c679"} Jul 24 00:12:20.869663 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.869032 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d7q6r" event={"ID":"1a539eef-502e-48e6-8020-2e6d9cc6ff5b","Type":"ContainerStarted","Data":"aaa6df38e5485fba3427437f41c8b2238648f3ab0509e7b797c98f46e9ee25a6"} Jul 24 00:12:20.874659 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.874588 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk" event={"ID":"1cf6076a-b8f1-44aa-a565-28328b56a8d9","Type":"ContainerStarted","Data":"1d4ea56874832ee2223acd39e7762ff0dd7f29a3f317bb44410e8730a2c4fabe"} Jul 24 00:12:20.875438 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.875392 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk" Jul 24 00:12:20.879072 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.878367 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-rdchn" podStartSLOduration=12.707705487 podStartE2EDuration="21.878347951s" totalImagesPullingTime="9.170642464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:11:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:20.876539873 +0000 UTC m=+69.481854198" watchObservedRunningTime="2026-07-24 00:12:20.878347951 +0000 UTC m=+69.483662268" Jul 24 00:12:20.882526 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.881814 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" event={"ID":"b164d179-d56a-451e-9b4e-00b001172f51","Type":"ContainerStarted","Data":"65d3649a6fbe3cb184166fc827e8896ff8952c2c829d356fd66efb96b4dc8bee"} Jul 24 00:12:20.882664 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.882559 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk" Jul 24 00:12:20.886395 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.886330 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2rr2t" event={"ID":"0b1308db-61d6-49a8-b546-cf5bff91afb5","Type":"ContainerStarted","Data":"026da9d209addd5850ca92dcada262aede05780e49cf6dd5742a275491e5352b"} Jul 24 00:12:20.890225 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.889437 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" event={"ID":"4acaa63f-fc09-4074-9806-f9e9a5b43142","Type":"ContainerStarted","Data":"15aec50551c9b323bad41d7a31b9bf42ddf355cdaab87c9792dc78b5c2d9a3e8"} Jul 24 00:12:20.890225 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.889493 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" event={"ID":"4acaa63f-fc09-4074-9806-f9e9a5b43142","Type":"ContainerStarted","Data":"7aa6c2a18e642a64e2e30976c7fe95147d1e17c1dec9bbd64471f16a132df25b"} Jul 24 00:12:20.890225 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.889521 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" event={"ID":"4acaa63f-fc09-4074-9806-f9e9a5b43142","Type":"ContainerStarted","Data":"2328cd5c1b58888b3a1c1b486de24253f5c28a0a7781abb8588ad4a48a53c277"} Jul 24 00:12:20.892903 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.892881 2560 generic.go:356] "Generic (PLEG): container finished" podID="2f0a1517-451f-41a0-91b8-382111995fff" containerID="755fb24d768c5fa0dc72d61d6120dab6c63cd6c2aa36c9e2e1f434cd37135eb4" exitCode=0 Jul 24 00:12:20.893225 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.893029 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f0a1517-451f-41a0-91b8-382111995fff","Type":"ContainerDied","Data":"755fb24d768c5fa0dc72d61d6120dab6c63cd6c2aa36c9e2e1f434cd37135eb4"} Jul 24 00:12:20.897383 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.897339 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-d7q6r" podStartSLOduration=3.926387666 podStartE2EDuration="35.897320267s" totalImagesPullingTime="31.970932601s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 00:11:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:20.896409587 +0000 UTC m=+69.501723904" watchObservedRunningTime="2026-07-24 00:12:20.897320267 +0000 UTC m=+69.502634581" Jul 24 00:12:20.901619 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.901591 2560 generic.go:356] "Generic (PLEG): container finished" podID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerID="39a064402fdf1f08a2d77454e9f28e7ff6245931074bfd0e33f15007ac9db3bb" exitCode=0 Jul 24 00:12:20.901753 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.901705 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565","Type":"ContainerDied","Data":"39a064402fdf1f08a2d77454e9f28e7ff6245931074bfd0e33f15007ac9db3bb"} Jul 24 00:12:20.908447 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.908420 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" event={"ID":"6819273d-e59d-4321-b9f3-e3afd59cf648","Type":"ContainerStarted","Data":"9f46e7e38d0ac77038ffc898a68ac13df04726f9e505d597a639918de80ca5c9"} Jul 24 00:12:20.908917 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.908670 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" event={"ID":"6819273d-e59d-4321-b9f3-e3afd59cf648","Type":"ContainerStarted","Data":"6e8955f0399fb3c1c93151e3564329151f474795afa1f99edf961238cb797a22"} Jul 24 00:12:20.909168 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.909148 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" event={"ID":"6819273d-e59d-4321-b9f3-e3afd59cf648","Type":"ContainerStarted","Data":"aca9dfb7e6687cceeef235eca9cfa7832ba5f59c64a46f6b577a5c774fdda7ed"} Jul 24 00:12:20.940713 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.939880 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-nhxgt" podStartSLOduration=12.660757918 podStartE2EDuration="21.939856776s" totalImagesPullingTime="9.279098858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:11:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:20.938543817 +0000 UTC m=+69.543858172" watchObservedRunningTime="2026-07-24 00:12:20.939856776 +0000 UTC m=+69.545171094" Jul 24 00:12:20.940713 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:20.940241 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rvvkk" podStartSLOduration=8.635102966 podStartE2EDuration="17.940234068s" totalImagesPullingTime="9.305131102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:12:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:20.9161357 +0000 UTC m=+69.521450017" watchObservedRunningTime="2026-07-24 00:12:20.940234068 +0000 UTC m=+69.545548384" Jul 24 00:12:21.036527 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:21.036286 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" podStartSLOduration=8.725542236 podStartE2EDuration="18.036263294s" totalImagesPullingTime="9.310721058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:12:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:21.035512476 +0000 UTC m=+69.640826789" watchObservedRunningTime="2026-07-24 00:12:21.036263294 +0000 UTC m=+69.641577611" Jul 24 00:12:22.926587 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:22.926540 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" event={"ID":"6819273d-e59d-4321-b9f3-e3afd59cf648","Type":"ContainerStarted","Data":"2dfb9a6c273c524eabeb027aabb745cc2cf1ad78236c15128ba6b345b170282a"} Jul 24 00:12:22.927142 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:22.926658 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" event={"ID":"6819273d-e59d-4321-b9f3-e3afd59cf648","Type":"ContainerStarted","Data":"7b9994cf7603c03660d7f237905a353b3d50321601e81fec6fe76d5daf01922a"} Jul 24 00:12:22.927142 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:22.926704 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" event={"ID":"6819273d-e59d-4321-b9f3-e3afd59cf648","Type":"ContainerStarted","Data":"c61b43f5bcd39d6b60e843e2072ac2335762ec70dea6e87b61f20fefd84a8e05"} Jul 24 00:12:22.927378 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:22.927360 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:22.931120 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:22.931090 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lqb6d" event={"ID":"3266eb6f-f15e-4b80-ba25-b0f9807214e6","Type":"ContainerStarted","Data":"bc5a3e05809a4df5385f4171c90e2f3d8f502455dc7abd37a34db50999060716"} Jul 24 00:12:22.931250 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:22.931155 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lqb6d" event={"ID":"3266eb6f-f15e-4b80-ba25-b0f9807214e6","Type":"ContainerStarted","Data":"05a08943ba499afebcc7c16a9fa1f7ece1a80fde639ec1be3165d4d2bbb4140f"} Jul 24 00:12:22.978265 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:22.978199 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" podStartSLOduration=9.750061503 podStartE2EDuration="20.978174534s" totalImagesPullingTime="11.228113031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:12:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:22.957133148 +0000 UTC m=+71.562447465" watchObservedRunningTime="2026-07-24 00:12:22.978174534 +0000 UTC m=+71.583488829" Jul 24 00:12:22.979417 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:22.979375 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lqb6d" podStartSLOduration=67.664207414 podStartE2EDuration="1m9.979359584s" totalImagesPullingTime="2.31515217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:22.976425367 +0000 UTC m=+71.581739685" watchObservedRunningTime="2026-07-24 00:12:22.979359584 +0000 UTC m=+71.584673901" Jul 24 00:12:23.094287 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:23.094252 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:23.194555 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:23.194472 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:23.894365 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:23.894313 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:23.894557 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:23.894419 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:24.695003 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:24.694961 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:24.967210 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:24.967138 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:28.942545 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:28.942516 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6bcd874fcc-r2sb2" Jul 24 00:12:31.970280 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.970243 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k66hc" event={"ID":"ff0d6407-0175-4c0e-a54c-528dae80387a","Type":"ContainerStarted","Data":"43b99b5d6aef210880d9ef633246bb90f86485a21253d6a2c7b58e26f4c2aaf1"} Jul 24 00:12:31.971743 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.971718 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2rr2t" event={"ID":"0b1308db-61d6-49a8-b546-cf5bff91afb5","Type":"ContainerStarted","Data":"b5e5b7d1788710265e14e93ba803455dda5664bc5df777a1a183138eaac1b519"} Jul 24 00:12:31.971872 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.971794 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:12:31.974913 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.974885 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f0a1517-451f-41a0-91b8-382111995fff","Type":"ContainerStarted","Data":"a03ad8f8f0181bcba1c5d473dccb36d0313d9f81c855c267cb3f120f3c740728"} Jul 24 00:12:31.975023 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.974945 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f0a1517-451f-41a0-91b8-382111995fff","Type":"ContainerStarted","Data":"14f702400b95a3bc37f27bb562cd016c38f2f833f271e3e99f7b28709136cb15"} Jul 24 00:12:31.975023 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.975009 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f0a1517-451f-41a0-91b8-382111995fff","Type":"ContainerStarted","Data":"0d529c161d2c7ca7630a06ea9be6df565bce973a84742fa2c604d8e2981d175b"} Jul 24 00:12:31.975121 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.975038 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f0a1517-451f-41a0-91b8-382111995fff","Type":"ContainerStarted","Data":"5c0feadf068a4254ad09ad12f3d180236437dd9ac6df4ae2aa724208a5f5aa81"} Jul 24 00:12:31.975121 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.975066 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f0a1517-451f-41a0-91b8-382111995fff","Type":"ContainerStarted","Data":"2d943d93d08716a59163953bd01550d77a94ff4fe1c9aaf21e24177278f593b3"} Jul 24 00:12:31.975121 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.975100 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f0a1517-451f-41a0-91b8-382111995fff","Type":"ContainerStarted","Data":"3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac"} Jul 24 00:12:31.977974 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.977953 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565","Type":"ContainerStarted","Data":"3c9e8a2c696326f4122a4eaf27d5df29bd0897d3cccd98f70a82619f0e8bae41"} Jul 24 00:12:31.978090 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.978013 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565","Type":"ContainerStarted","Data":"1fd4419fdf2e635268802c18fe9b71d6aae0a324aaabeebd43276d049d066fbe"} Jul 24 00:12:31.978090 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.978059 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565","Type":"ContainerStarted","Data":"eb1f972341f612240aadc891fa8ebcca5eb545a6cfdf835946eea3c549060aa1"} Jul 24 00:12:31.978208 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.978097 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565","Type":"ContainerStarted","Data":"648824d5d02ba5792f4c322e9bdf99b9cc4e5822b867957b7b44ee72942b1710"} Jul 24 00:12:31.978208 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.978144 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565","Type":"ContainerStarted","Data":"6341c52a13129ed69139ae85b7da754877f7238f0c98e8f57a012a00a6efd7bd"} Jul 24 00:12:31.978208 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.978182 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565","Type":"ContainerStarted","Data":"eff8cf997af8e979a65feb37b59548125da35211c48f22ae65d56c02c83ba32a"} Jul 24 00:12:31.998032 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:31.997991 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k66hc" podStartSLOduration=68.083816221 podStartE2EDuration="1m18.997977331s" totalImagesPullingTime="10.91416111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 00:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:31.996962264 +0000 UTC m=+80.602276582" watchObservedRunningTime="2026-07-24 00:12:31.997977331 +0000 UTC m=+80.603291670" Jul 24 00:12:32.049994 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:32.049934 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=12.834582721 podStartE2EDuration="32.04991391s" totalImagesPullingTime="19.201229402s" totalInitContainerRuntime="14.101787ms" isStatefulPod=false podCreationTimestamp="2026-07-24 00:12:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:32.046765564 +0000 UTC m=+80.652079881" watchObservedRunningTime="2026-07-24 00:12:32.04991391 +0000 UTC m=+80.655228227" Jul 24 00:12:32.079213 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:32.079154 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.826668025 podStartE2EDuration="27.079132379s" totalImagesPullingTime="19.229015688s" totalInitContainerRuntime="23.448666ms" isStatefulPod=false podCreationTimestamp="2026-07-24 00:12:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:32.078465999 +0000 UTC m=+80.683780315" watchObservedRunningTime="2026-07-24 00:12:32.079132379 +0000 UTC m=+80.684446698" Jul 24 00:12:32.095869 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:32.095815 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2rr2t" podStartSLOduration=69.182152613 podStartE2EDuration="1m20.095795323s" totalImagesPullingTime="10.91364271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:32.094922559 +0000 UTC m=+80.700236886" watchObservedRunningTime="2026-07-24 00:12:32.095795323 +0000 UTC m=+80.701109640" Jul 24 00:12:33.833270 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:33.833148 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:33.886943 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:33.886915 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:33.987904 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:33.987864 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:34.003328 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:34.003299 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:39.911985 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:39.911947 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b895d5c5b-l9pv4" podUID="c572a5dd-23eb-496c-9835-3d565fe5631c" containerName="console" containerID="cri-o://6b1b9435a7960d775c7ba8a565f8e4381330ea8ac2bf91a6251aad84163b0b7a" gracePeriod=15 Jul 24 00:12:40.199190 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.199155 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:12:40.259477 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.259351 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c572a5dd-23eb-496c-9835-3d565fe5631c-console-serving-cert\") pod \"c572a5dd-23eb-496c-9835-3d565fe5631c\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " Jul 24 00:12:40.259477 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.259394 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c572a5dd-23eb-496c-9835-3d565fe5631c-console-oauth-config\") pod \"c572a5dd-23eb-496c-9835-3d565fe5631c\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " Jul 24 00:12:40.259477 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.259429 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptlpw\" (UniqueName: \"kubernetes.io/projected/c572a5dd-23eb-496c-9835-3d565fe5631c-kube-api-access-ptlpw\") pod \"c572a5dd-23eb-496c-9835-3d565fe5631c\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " Jul 24 00:12:40.259477 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.259457 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-oauth-serving-cert\") pod \"c572a5dd-23eb-496c-9835-3d565fe5631c\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " Jul 24 00:12:40.259848 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.259495 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-service-ca\") pod \"c572a5dd-23eb-496c-9835-3d565fe5631c\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " Jul 24 00:12:40.259848 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.259594 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-console-config\") pod \"c572a5dd-23eb-496c-9835-3d565fe5631c\" (UID: \"c572a5dd-23eb-496c-9835-3d565fe5631c\") " Jul 24 00:12:40.259948 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.259884 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c572a5dd-23eb-496c-9835-3d565fe5631c" (UID: "c572a5dd-23eb-496c-9835-3d565fe5631c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:12:40.260005 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.259976 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-service-ca" (OuterVolumeSpecName: "service-ca") pod "c572a5dd-23eb-496c-9835-3d565fe5631c" (UID: "c572a5dd-23eb-496c-9835-3d565fe5631c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:12:40.260158 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.260130 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-console-config" (OuterVolumeSpecName: "console-config") pod "c572a5dd-23eb-496c-9835-3d565fe5631c" (UID: "c572a5dd-23eb-496c-9835-3d565fe5631c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:12:40.261941 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.261911 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c572a5dd-23eb-496c-9835-3d565fe5631c-kube-api-access-ptlpw" (OuterVolumeSpecName: "kube-api-access-ptlpw") pod "c572a5dd-23eb-496c-9835-3d565fe5631c" (UID: "c572a5dd-23eb-496c-9835-3d565fe5631c"). InnerVolumeSpecName "kube-api-access-ptlpw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:12:40.262066 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.262008 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c572a5dd-23eb-496c-9835-3d565fe5631c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c572a5dd-23eb-496c-9835-3d565fe5631c" (UID: "c572a5dd-23eb-496c-9835-3d565fe5631c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:40.262346 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.262328 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c572a5dd-23eb-496c-9835-3d565fe5631c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c572a5dd-23eb-496c-9835-3d565fe5631c" (UID: "c572a5dd-23eb-496c-9835-3d565fe5631c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:40.360248 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.360221 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-console-config\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:40.360248 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.360246 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c572a5dd-23eb-496c-9835-3d565fe5631c-console-serving-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:40.360413 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.360256 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c572a5dd-23eb-496c-9835-3d565fe5631c-console-oauth-config\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:40.360413 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.360265 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptlpw\" (UniqueName: \"kubernetes.io/projected/c572a5dd-23eb-496c-9835-3d565fe5631c-kube-api-access-ptlpw\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:40.360413 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.360274 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-oauth-serving-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:40.360413 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:40.360284 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c572a5dd-23eb-496c-9835-3d565fe5631c-service-ca\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:41.017651 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:41.017617 2560 generic.go:356] "Generic (PLEG): container finished" podID="c572a5dd-23eb-496c-9835-3d565fe5631c" containerID="6b1b9435a7960d775c7ba8a565f8e4381330ea8ac2bf91a6251aad84163b0b7a" exitCode=0 Jul 24 00:12:41.018216 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:41.017675 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b895d5c5b-l9pv4" Jul 24 00:12:41.018216 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:41.017711 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b895d5c5b-l9pv4" event={"ID":"c572a5dd-23eb-496c-9835-3d565fe5631c","Type":"ContainerDied","Data":"6b1b9435a7960d775c7ba8a565f8e4381330ea8ac2bf91a6251aad84163b0b7a"} Jul 24 00:12:41.018216 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:41.017777 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b895d5c5b-l9pv4" event={"ID":"c572a5dd-23eb-496c-9835-3d565fe5631c","Type":"ContainerDied","Data":"b83edc0c05d2bdfb493c37a2623392010b3362745d5dbd665f2de4aee1503966"} Jul 24 00:12:41.018216 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:41.017810 2560 scope.go:117] "RemoveContainer" containerID="6b1b9435a7960d775c7ba8a565f8e4381330ea8ac2bf91a6251aad84163b0b7a" Jul 24 00:12:41.032923 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:41.032905 2560 scope.go:117] "RemoveContainer" containerID="6b1b9435a7960d775c7ba8a565f8e4381330ea8ac2bf91a6251aad84163b0b7a" Jul 24 00:12:41.033250 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:41.033223 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b1b9435a7960d775c7ba8a565f8e4381330ea8ac2bf91a6251aad84163b0b7a\": container with ID starting with 6b1b9435a7960d775c7ba8a565f8e4381330ea8ac2bf91a6251aad84163b0b7a not found: ID does not exist" containerID="6b1b9435a7960d775c7ba8a565f8e4381330ea8ac2bf91a6251aad84163b0b7a" Jul 24 00:12:41.033335 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:41.033271 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b1b9435a7960d775c7ba8a565f8e4381330ea8ac2bf91a6251aad84163b0b7a"} err="failed to get container status \"6b1b9435a7960d775c7ba8a565f8e4381330ea8ac2bf91a6251aad84163b0b7a\": rpc error: code = NotFound desc = could not find container \"6b1b9435a7960d775c7ba8a565f8e4381330ea8ac2bf91a6251aad84163b0b7a\": container with ID starting with 6b1b9435a7960d775c7ba8a565f8e4381330ea8ac2bf91a6251aad84163b0b7a not found: ID does not exist" Jul 24 00:12:41.039820 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:41.039788 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b895d5c5b-l9pv4"] Jul 24 00:12:41.044040 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:41.044017 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b895d5c5b-l9pv4"] Jul 24 00:12:42.348704 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:42.348672 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c572a5dd-23eb-496c-9835-3d565fe5631c" path="/var/lib/kubelet/pods/c572a5dd-23eb-496c-9835-3d565fe5631c/volumes" Jul 24 00:12:43.900268 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:43.900240 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:43.904446 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:43.904415 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6d8f567488-7cnck" Jul 24 00:12:47.812420 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:47.812314 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:48.009036 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:48.009000 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:48.072887 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:48.072795 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:48.385052 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:48.384969 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:48.847036 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:48.847003 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 00:12:48.847429 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:48.847280 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="alertmanager" containerID="cri-o://eff8cf997af8e979a65feb37b59548125da35211c48f22ae65d56c02c83ba32a" gracePeriod=120 Jul 24 00:12:48.847429 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:48.847339 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="kube-rbac-proxy-web" containerID="cri-o://648824d5d02ba5792f4c322e9bdf99b9cc4e5822b867957b7b44ee72942b1710" gracePeriod=120 Jul 24 00:12:48.847429 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:48.847339 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="kube-rbac-proxy-metric" containerID="cri-o://1fd4419fdf2e635268802c18fe9b71d6aae0a324aaabeebd43276d049d066fbe" gracePeriod=120 Jul 24 00:12:48.847429 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:48.847374 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="kube-rbac-proxy" containerID="cri-o://eb1f972341f612240aadc891fa8ebcca5eb545a6cfdf835946eea3c549060aa1" gracePeriod=120 Jul 24 00:12:48.847429 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:48.847426 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="prom-label-proxy" containerID="cri-o://3c9e8a2c696326f4122a4eaf27d5df29bd0897d3cccd98f70a82619f0e8bae41" gracePeriod=120 Jul 24 00:12:48.847675 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:48.847411 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="config-reloader" containerID="cri-o://6341c52a13129ed69139ae85b7da754877f7238f0c98e8f57a012a00a6efd7bd" gracePeriod=120 Jul 24 00:12:49.058580 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.058554 2560 generic.go:356] "Generic (PLEG): container finished" podID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerID="3c9e8a2c696326f4122a4eaf27d5df29bd0897d3cccd98f70a82619f0e8bae41" exitCode=0 Jul 24 00:12:49.058580 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.058575 2560 generic.go:356] "Generic (PLEG): container finished" podID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerID="1fd4419fdf2e635268802c18fe9b71d6aae0a324aaabeebd43276d049d066fbe" exitCode=0 Jul 24 00:12:49.058580 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.058581 2560 generic.go:356] "Generic (PLEG): container finished" podID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerID="eb1f972341f612240aadc891fa8ebcca5eb545a6cfdf835946eea3c549060aa1" exitCode=0 Jul 24 00:12:49.058580 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.058587 2560 generic.go:356] "Generic (PLEG): container finished" podID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerID="648824d5d02ba5792f4c322e9bdf99b9cc4e5822b867957b7b44ee72942b1710" exitCode=0 Jul 24 00:12:49.058826 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.058592 2560 generic.go:356] "Generic (PLEG): container finished" podID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerID="6341c52a13129ed69139ae85b7da754877f7238f0c98e8f57a012a00a6efd7bd" exitCode=0 Jul 24 00:12:49.058826 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.058630 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565","Type":"ContainerDied","Data":"3c9e8a2c696326f4122a4eaf27d5df29bd0897d3cccd98f70a82619f0e8bae41"} Jul 24 00:12:49.058826 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.058736 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565","Type":"ContainerDied","Data":"1fd4419fdf2e635268802c18fe9b71d6aae0a324aaabeebd43276d049d066fbe"} Jul 24 00:12:49.058826 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.058784 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565","Type":"ContainerDied","Data":"eb1f972341f612240aadc891fa8ebcca5eb545a6cfdf835946eea3c549060aa1"} Jul 24 00:12:49.058826 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.058825 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565","Type":"ContainerDied","Data":"648824d5d02ba5792f4c322e9bdf99b9cc4e5822b867957b7b44ee72942b1710"} Jul 24 00:12:49.058975 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.058868 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565","Type":"ContainerDied","Data":"6341c52a13129ed69139ae85b7da754877f7238f0c98e8f57a012a00a6efd7bd"} Jul 24 00:12:49.630964 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.630929 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:49.750732 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.750706 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy-metric\") pod \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " Jul 24 00:12:49.750889 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.750739 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy-web\") pod \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " Jul 24 00:12:49.750889 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.750776 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-web-config\") pod \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " Jul 24 00:12:49.750889 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.750805 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-cluster-tls-config\") pod \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " Jul 24 00:12:49.751048 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.751013 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-alertmanager-trusted-ca-bundle\") pod \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " Jul 24 00:12:49.751190 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.751069 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-tls-assets\") pod \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " Jul 24 00:12:49.751190 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.751098 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-alertmanager-main-db\") pod \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " Jul 24 00:12:49.751190 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.751154 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy\") pod \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " Jul 24 00:12:49.751347 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.751202 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-main-tls\") pod \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " Jul 24 00:12:49.751347 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.751227 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-config-volume\") pod \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " Jul 24 00:12:49.751347 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.751258 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-metrics-client-ca\") pod \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " Jul 24 00:12:49.751347 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.751311 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-config-out\") pod \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " Jul 24 00:12:49.751347 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.751344 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrrvd\" (UniqueName: \"kubernetes.io/projected/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-kube-api-access-hrrvd\") pod \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\" (UID: \"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565\") " Jul 24 00:12:49.751589 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.751358 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" (UID: "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:12:49.751589 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.751569 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" (UID: "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:12:49.751766 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.751680 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-alertmanager-main-db\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:49.751766 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.751701 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:49.752169 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.752143 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" (UID: "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:12:49.754613 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.754572 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" (UID: "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:49.754760 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.754719 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-kube-api-access-hrrvd" (OuterVolumeSpecName: "kube-api-access-hrrvd") pod "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" (UID: "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565"). InnerVolumeSpecName "kube-api-access-hrrvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:12:49.754931 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.754907 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" (UID: "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:49.755019 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.754978 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" (UID: "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:12:49.755080 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.755036 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" (UID: "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:49.755161 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.755137 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-config-volume" (OuterVolumeSpecName: "config-volume") pod "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" (UID: "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:49.755249 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.755229 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-config-out" (OuterVolumeSpecName: "config-out") pod "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" (UID: "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:12:49.755714 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.755687 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" (UID: "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:49.758544 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.758524 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" (UID: "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:49.764889 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.764863 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-web-config" (OuterVolumeSpecName: "web-config") pod "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" (UID: "8880ee1f-1a6e-4a69-82c2-b3e9be9ae565"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:49.852523 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.852486 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrrvd\" (UniqueName: \"kubernetes.io/projected/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-kube-api-access-hrrvd\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:49.852523 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.852517 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:49.852523 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.852529 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:49.852989 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.852538 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-web-config\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:49.852989 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.852547 2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-cluster-tls-config\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:49.852989 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.852556 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-tls-assets\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:49.852989 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.852564 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:49.852989 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.852573 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-secret-alertmanager-main-tls\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:49.852989 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.852581 2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:49.852989 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.852589 2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-metrics-client-ca\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:49.852989 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:49.852598 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565-config-out\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:50.066451 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.066374 2560 generic.go:356] "Generic (PLEG): container finished" podID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerID="eff8cf997af8e979a65feb37b59548125da35211c48f22ae65d56c02c83ba32a" exitCode=0 Jul 24 00:12:50.066451 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.066437 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565","Type":"ContainerDied","Data":"eff8cf997af8e979a65feb37b59548125da35211c48f22ae65d56c02c83ba32a"} Jul 24 00:12:50.066627 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.066455 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.066627 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.066496 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8880ee1f-1a6e-4a69-82c2-b3e9be9ae565","Type":"ContainerDied","Data":"5338279c690ea0abad0ec161e5557d4e297db9a96133aacc6ffb698a55a0f791"} Jul 24 00:12:50.066627 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.066528 2560 scope.go:117] "RemoveContainer" containerID="3c9e8a2c696326f4122a4eaf27d5df29bd0897d3cccd98f70a82619f0e8bae41" Jul 24 00:12:50.076145 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.076116 2560 scope.go:117] "RemoveContainer" containerID="1fd4419fdf2e635268802c18fe9b71d6aae0a324aaabeebd43276d049d066fbe" Jul 24 00:12:50.083712 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.083693 2560 scope.go:117] "RemoveContainer" containerID="eb1f972341f612240aadc891fa8ebcca5eb545a6cfdf835946eea3c549060aa1" Jul 24 00:12:50.090449 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.090432 2560 scope.go:117] "RemoveContainer" containerID="648824d5d02ba5792f4c322e9bdf99b9cc4e5822b867957b7b44ee72942b1710" Jul 24 00:12:50.097339 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.097321 2560 scope.go:117] "RemoveContainer" containerID="6341c52a13129ed69139ae85b7da754877f7238f0c98e8f57a012a00a6efd7bd" Jul 24 00:12:50.103716 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.103693 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 00:12:50.105577 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.105557 2560 scope.go:117] "RemoveContainer" containerID="eff8cf997af8e979a65feb37b59548125da35211c48f22ae65d56c02c83ba32a" Jul 24 00:12:50.108446 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.108419 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 00:12:50.113498 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.113474 2560 scope.go:117] "RemoveContainer" containerID="39a064402fdf1f08a2d77454e9f28e7ff6245931074bfd0e33f15007ac9db3bb" Jul 24 00:12:50.120605 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.120589 2560 scope.go:117] "RemoveContainer" containerID="3c9e8a2c696326f4122a4eaf27d5df29bd0897d3cccd98f70a82619f0e8bae41" Jul 24 00:12:50.120924 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:50.120904 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c9e8a2c696326f4122a4eaf27d5df29bd0897d3cccd98f70a82619f0e8bae41\": container with ID starting with 3c9e8a2c696326f4122a4eaf27d5df29bd0897d3cccd98f70a82619f0e8bae41 not found: ID does not exist" containerID="3c9e8a2c696326f4122a4eaf27d5df29bd0897d3cccd98f70a82619f0e8bae41" Jul 24 00:12:50.120985 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.120931 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c9e8a2c696326f4122a4eaf27d5df29bd0897d3cccd98f70a82619f0e8bae41"} err="failed to get container status \"3c9e8a2c696326f4122a4eaf27d5df29bd0897d3cccd98f70a82619f0e8bae41\": rpc error: code = NotFound desc = could not find container \"3c9e8a2c696326f4122a4eaf27d5df29bd0897d3cccd98f70a82619f0e8bae41\": container with ID starting with 3c9e8a2c696326f4122a4eaf27d5df29bd0897d3cccd98f70a82619f0e8bae41 not found: ID does not exist" Jul 24 00:12:50.120985 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.120949 2560 scope.go:117] "RemoveContainer" containerID="1fd4419fdf2e635268802c18fe9b71d6aae0a324aaabeebd43276d049d066fbe" Jul 24 00:12:50.122127 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:50.121992 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fd4419fdf2e635268802c18fe9b71d6aae0a324aaabeebd43276d049d066fbe\": container with ID starting with 1fd4419fdf2e635268802c18fe9b71d6aae0a324aaabeebd43276d049d066fbe not found: ID does not exist" containerID="1fd4419fdf2e635268802c18fe9b71d6aae0a324aaabeebd43276d049d066fbe" Jul 24 00:12:50.122127 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.122025 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fd4419fdf2e635268802c18fe9b71d6aae0a324aaabeebd43276d049d066fbe"} err="failed to get container status \"1fd4419fdf2e635268802c18fe9b71d6aae0a324aaabeebd43276d049d066fbe\": rpc error: code = NotFound desc = could not find container \"1fd4419fdf2e635268802c18fe9b71d6aae0a324aaabeebd43276d049d066fbe\": container with ID starting with 1fd4419fdf2e635268802c18fe9b71d6aae0a324aaabeebd43276d049d066fbe not found: ID does not exist" Jul 24 00:12:50.122127 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.122047 2560 scope.go:117] "RemoveContainer" containerID="eb1f972341f612240aadc891fa8ebcca5eb545a6cfdf835946eea3c549060aa1" Jul 24 00:12:50.123677 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:50.123627 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb1f972341f612240aadc891fa8ebcca5eb545a6cfdf835946eea3c549060aa1\": container with ID starting with eb1f972341f612240aadc891fa8ebcca5eb545a6cfdf835946eea3c549060aa1 not found: ID does not exist" containerID="eb1f972341f612240aadc891fa8ebcca5eb545a6cfdf835946eea3c549060aa1" Jul 24 00:12:50.123763 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.123677 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb1f972341f612240aadc891fa8ebcca5eb545a6cfdf835946eea3c549060aa1"} err="failed to get container status \"eb1f972341f612240aadc891fa8ebcca5eb545a6cfdf835946eea3c549060aa1\": rpc error: code = NotFound desc = could not find container \"eb1f972341f612240aadc891fa8ebcca5eb545a6cfdf835946eea3c549060aa1\": container with ID starting with eb1f972341f612240aadc891fa8ebcca5eb545a6cfdf835946eea3c549060aa1 not found: ID does not exist" Jul 24 00:12:50.123763 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.123692 2560 scope.go:117] "RemoveContainer" containerID="648824d5d02ba5792f4c322e9bdf99b9cc4e5822b867957b7b44ee72942b1710" Jul 24 00:12:50.124258 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:50.124239 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"648824d5d02ba5792f4c322e9bdf99b9cc4e5822b867957b7b44ee72942b1710\": container with ID starting with 648824d5d02ba5792f4c322e9bdf99b9cc4e5822b867957b7b44ee72942b1710 not found: ID does not exist" containerID="648824d5d02ba5792f4c322e9bdf99b9cc4e5822b867957b7b44ee72942b1710" Jul 24 00:12:50.124335 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.124272 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"648824d5d02ba5792f4c322e9bdf99b9cc4e5822b867957b7b44ee72942b1710"} err="failed to get container status \"648824d5d02ba5792f4c322e9bdf99b9cc4e5822b867957b7b44ee72942b1710\": rpc error: code = NotFound desc = could not find container \"648824d5d02ba5792f4c322e9bdf99b9cc4e5822b867957b7b44ee72942b1710\": container with ID starting with 648824d5d02ba5792f4c322e9bdf99b9cc4e5822b867957b7b44ee72942b1710 not found: ID does not exist" Jul 24 00:12:50.124335 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.124288 2560 scope.go:117] "RemoveContainer" containerID="6341c52a13129ed69139ae85b7da754877f7238f0c98e8f57a012a00a6efd7bd" Jul 24 00:12:50.124553 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:50.124533 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6341c52a13129ed69139ae85b7da754877f7238f0c98e8f57a012a00a6efd7bd\": container with ID starting with 6341c52a13129ed69139ae85b7da754877f7238f0c98e8f57a012a00a6efd7bd not found: ID does not exist" containerID="6341c52a13129ed69139ae85b7da754877f7238f0c98e8f57a012a00a6efd7bd" Jul 24 00:12:50.124602 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.124563 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6341c52a13129ed69139ae85b7da754877f7238f0c98e8f57a012a00a6efd7bd"} err="failed to get container status \"6341c52a13129ed69139ae85b7da754877f7238f0c98e8f57a012a00a6efd7bd\": rpc error: code = NotFound desc = could not find container \"6341c52a13129ed69139ae85b7da754877f7238f0c98e8f57a012a00a6efd7bd\": container with ID starting with 6341c52a13129ed69139ae85b7da754877f7238f0c98e8f57a012a00a6efd7bd not found: ID does not exist" Jul 24 00:12:50.124602 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.124585 2560 scope.go:117] "RemoveContainer" containerID="eff8cf997af8e979a65feb37b59548125da35211c48f22ae65d56c02c83ba32a" Jul 24 00:12:50.124860 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:50.124839 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eff8cf997af8e979a65feb37b59548125da35211c48f22ae65d56c02c83ba32a\": container with ID starting with eff8cf997af8e979a65feb37b59548125da35211c48f22ae65d56c02c83ba32a not found: ID does not exist" containerID="eff8cf997af8e979a65feb37b59548125da35211c48f22ae65d56c02c83ba32a" Jul 24 00:12:50.124951 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.124867 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eff8cf997af8e979a65feb37b59548125da35211c48f22ae65d56c02c83ba32a"} err="failed to get container status \"eff8cf997af8e979a65feb37b59548125da35211c48f22ae65d56c02c83ba32a\": rpc error: code = NotFound desc = could not find container \"eff8cf997af8e979a65feb37b59548125da35211c48f22ae65d56c02c83ba32a\": container with ID starting with eff8cf997af8e979a65feb37b59548125da35211c48f22ae65d56c02c83ba32a not found: ID does not exist" Jul 24 00:12:50.124951 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.124888 2560 scope.go:117] "RemoveContainer" containerID="39a064402fdf1f08a2d77454e9f28e7ff6245931074bfd0e33f15007ac9db3bb" Jul 24 00:12:50.125113 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:50.125099 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39a064402fdf1f08a2d77454e9f28e7ff6245931074bfd0e33f15007ac9db3bb\": container with ID starting with 39a064402fdf1f08a2d77454e9f28e7ff6245931074bfd0e33f15007ac9db3bb not found: ID does not exist" containerID="39a064402fdf1f08a2d77454e9f28e7ff6245931074bfd0e33f15007ac9db3bb" Jul 24 00:12:50.125154 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.125117 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39a064402fdf1f08a2d77454e9f28e7ff6245931074bfd0e33f15007ac9db3bb"} err="failed to get container status \"39a064402fdf1f08a2d77454e9f28e7ff6245931074bfd0e33f15007ac9db3bb\": rpc error: code = NotFound desc = could not find container \"39a064402fdf1f08a2d77454e9f28e7ff6245931074bfd0e33f15007ac9db3bb\": container with ID starting with 39a064402fdf1f08a2d77454e9f28e7ff6245931074bfd0e33f15007ac9db3bb not found: ID does not exist" Jul 24 00:12:50.136868 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.136846 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 00:12:50.137201 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137187 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="alertmanager" Jul 24 00:12:50.137201 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137203 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="alertmanager" Jul 24 00:12:50.137283 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137211 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="config-reloader" Jul 24 00:12:50.137283 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137216 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="config-reloader" Jul 24 00:12:50.137283 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137224 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="prom-label-proxy" Jul 24 00:12:50.137283 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137231 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="prom-label-proxy" Jul 24 00:12:50.137283 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137261 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="init-config-reloader" Jul 24 00:12:50.137283 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137268 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="init-config-reloader" Jul 24 00:12:50.137283 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137274 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c572a5dd-23eb-496c-9835-3d565fe5631c" containerName="console" Jul 24 00:12:50.137283 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137279 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c572a5dd-23eb-496c-9835-3d565fe5631c" containerName="console" Jul 24 00:12:50.137283 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137285 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="kube-rbac-proxy-web" Jul 24 00:12:50.137519 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137291 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="kube-rbac-proxy-web" Jul 24 00:12:50.137519 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137299 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="kube-rbac-proxy" Jul 24 00:12:50.137519 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137304 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="kube-rbac-proxy" Jul 24 00:12:50.137519 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137309 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="kube-rbac-proxy-metric" Jul 24 00:12:50.137519 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137316 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="kube-rbac-proxy-metric" Jul 24 00:12:50.137519 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137364 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="config-reloader" Jul 24 00:12:50.137519 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137371 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c572a5dd-23eb-496c-9835-3d565fe5631c" containerName="console" Jul 24 00:12:50.137519 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137378 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="prom-label-proxy" Jul 24 00:12:50.137519 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137383 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="kube-rbac-proxy" Jul 24 00:12:50.137519 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137388 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="kube-rbac-proxy-metric" Jul 24 00:12:50.137519 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137393 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="kube-rbac-proxy-web" Jul 24 00:12:50.137519 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.137400 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" containerName="alertmanager" Jul 24 00:12:50.157672 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.157628 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.158542 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.158522 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 00:12:50.160942 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.160758 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 24 00:12:50.160942 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.160804 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 24 00:12:50.160942 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.160805 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 24 00:12:50.160942 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.160836 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 24 00:12:50.160942 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.160858 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 24 00:12:50.160942 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.160805 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 24 00:12:50.161256 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.161237 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-j2cs9\"" Jul 24 00:12:50.161313 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.161291 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 24 00:12:50.161365 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.161350 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 24 00:12:50.166775 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.166756 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 24 00:12:50.256908 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.256876 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.257087 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.256919 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.257087 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.256947 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.257087 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.257028 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpk6r\" (UniqueName: \"kubernetes.io/projected/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-kube-api-access-vpk6r\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.257087 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.257071 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.257219 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.257096 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.257219 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.257113 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.257219 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.257136 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.257219 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.257172 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-web-config\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.257331 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.257235 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.257331 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.257257 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.257331 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.257283 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-config-volume\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.257331 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.257301 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-config-out\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.346905 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.346831 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8880ee1f-1a6e-4a69-82c2-b3e9be9ae565" path="/var/lib/kubelet/pods/8880ee1f-1a6e-4a69-82c2-b3e9be9ae565/volumes" Jul 24 00:12:50.358003 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.357981 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.358146 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.358009 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-web-config\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.358146 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.358030 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.358146 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.358048 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.358146 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.358068 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-config-volume\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.358146 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.358092 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-config-out\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.358146 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.358144 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.358437 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.358175 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.358437 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.358192 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.358437 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.358237 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpk6r\" (UniqueName: \"kubernetes.io/projected/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-kube-api-access-vpk6r\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.358437 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.358274 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.358437 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.358299 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.358437 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.358317 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.358751 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.358445 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.359576 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.358889 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.359874 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.359827 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.360969 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.360932 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-config-out\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.361234 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.361212 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.361715 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.361687 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.362141 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.362121 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-web-config\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.362141 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.362132 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.362260 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.362162 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.362260 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.362195 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.362260 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.362212 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-config-volume\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.363045 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.363030 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.368089 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.368072 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpk6r\" (UniqueName: \"kubernetes.io/projected/7d6efdc7-3622-4000-84a4-bfd29e1bdc45-kube-api-access-vpk6r\") pod \"alertmanager-main-0\" (UID: \"7d6efdc7-3622-4000-84a4-bfd29e1bdc45\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.468330 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.468292 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 00:12:50.633336 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:50.633316 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 00:12:50.635486 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:12:50.635456 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d6efdc7_3622_4000_84a4_bfd29e1bdc45.slice/crio-ef5346db1163050d77137890786627126eecf9a4a042c834ed12effdf303b7a3 WatchSource:0}: Error finding container ef5346db1163050d77137890786627126eecf9a4a042c834ed12effdf303b7a3: Status 404 returned error can't find the container with id ef5346db1163050d77137890786627126eecf9a4a042c834ed12effdf303b7a3 Jul 24 00:12:51.072404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:51.072374 2560 generic.go:356] "Generic (PLEG): container finished" podID="7d6efdc7-3622-4000-84a4-bfd29e1bdc45" containerID="1c0bc236f3d69fb22f602da5717e1f05972bb007ccc8a9e14dc82c469eceeaf5" exitCode=0 Jul 24 00:12:51.072791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:51.072441 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d6efdc7-3622-4000-84a4-bfd29e1bdc45","Type":"ContainerDied","Data":"1c0bc236f3d69fb22f602da5717e1f05972bb007ccc8a9e14dc82c469eceeaf5"} Jul 24 00:12:51.072791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:51.072491 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d6efdc7-3622-4000-84a4-bfd29e1bdc45","Type":"ContainerStarted","Data":"ef5346db1163050d77137890786627126eecf9a4a042c834ed12effdf303b7a3"} Jul 24 00:12:52.079977 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.079943 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d6efdc7-3622-4000-84a4-bfd29e1bdc45","Type":"ContainerStarted","Data":"10524e5755bc270b4cbcdfcc87b0bca68484f6d68e8c17a53e5e3a8b2d016d61"} Jul 24 00:12:52.080375 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.080035 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d6efdc7-3622-4000-84a4-bfd29e1bdc45","Type":"ContainerStarted","Data":"6c6b6103fd32febea43bdd575caf552a3b37325ef271ea122caf58a1b6cc01d3"} Jul 24 00:12:52.080375 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.080064 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d6efdc7-3622-4000-84a4-bfd29e1bdc45","Type":"ContainerStarted","Data":"3b52919411addc03a7c854f649d54e2bf21d2138d99912c626262efe767fa1c3"} Jul 24 00:12:52.080375 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.080090 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d6efdc7-3622-4000-84a4-bfd29e1bdc45","Type":"ContainerStarted","Data":"cd2d379b4f9358d5f0534923c16fbc979971c4855b3c7eae814b7a6e9a6c6acb"} Jul 24 00:12:52.080375 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.080113 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d6efdc7-3622-4000-84a4-bfd29e1bdc45","Type":"ContainerStarted","Data":"3090fc58b17a81f77c6aed418c3520bb0bc4dfc7e32418d1d095ff95dfc030ea"} Jul 24 00:12:52.080375 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.080137 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d6efdc7-3622-4000-84a4-bfd29e1bdc45","Type":"ContainerStarted","Data":"ca58c72403454a48023f184dab48e8aa962e8767a9d8d30c401c7517912db3e2"} Jul 24 00:12:52.109381 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.109303 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.088970366 podStartE2EDuration="2.109286493s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.316127ms" isStatefulPod=false podCreationTimestamp="2026-07-24 00:12:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:52.107635166 +0000 UTC m=+100.712949523" watchObservedRunningTime="2026-07-24 00:12:52.109286493 +0000 UTC m=+100.714600808" Jul 24 00:12:52.919204 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.919169 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-577b855d68-tp4ph"] Jul 24 00:12:52.974512 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.974472 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-577b855d68-tp4ph"] Jul 24 00:12:52.974684 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.974619 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:52.977603 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.977575 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 24 00:12:52.977713 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.977583 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 24 00:12:52.977766 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.977752 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-v5jmb\"" Jul 24 00:12:52.977830 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.977757 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 24 00:12:52.977830 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.977794 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 24 00:12:52.978002 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.977986 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 24 00:12:52.983444 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:52.983425 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 24 00:12:53.083015 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.082983 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-secret-telemeter-client\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.083408 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.083030 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.083408 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.083050 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-metrics-client-ca\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.083408 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.083071 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj6q7\" (UniqueName: \"kubernetes.io/projected/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-kube-api-access-zj6q7\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.083408 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.083188 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-federate-client-tls\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.083408 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.083233 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-serving-certs-ca-bundle\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.083408 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.083300 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-telemeter-client-tls\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.083408 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.083323 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-telemeter-trusted-ca-bundle\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.184194 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.184104 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-federate-client-tls\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.184194 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.184189 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-serving-certs-ca-bundle\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.184423 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.184245 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-telemeter-client-tls\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.184423 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.184294 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-telemeter-trusted-ca-bundle\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.184423 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.184379 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-secret-telemeter-client\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.184562 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.184476 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.184562 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.184510 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-metrics-client-ca\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.184562 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.184543 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj6q7\" (UniqueName: \"kubernetes.io/projected/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-kube-api-access-zj6q7\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.185725 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.185696 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-metrics-client-ca\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.186049 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.186018 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-serving-certs-ca-bundle\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.192720 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.189521 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-telemeter-trusted-ca-bundle\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.193663 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.193615 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-federate-client-tls\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.193922 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.193880 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-secret-telemeter-client\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.194001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.193970 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.197855 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.197833 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-telemeter-client-tls\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.198499 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.198479 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj6q7\" (UniqueName: \"kubernetes.io/projected/f54ac913-2a49-4d70-adfc-2fa67aadf7ed-kube-api-access-zj6q7\") pod \"telemeter-client-577b855d68-tp4ph\" (UID: \"f54ac913-2a49-4d70-adfc-2fa67aadf7ed\") " pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.205115 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.205090 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 00:12:53.205516 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.205467 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="prometheus" containerID="cri-o://3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac" gracePeriod=600 Jul 24 00:12:53.205618 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.205514 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="kube-rbac-proxy" containerID="cri-o://14f702400b95a3bc37f27bb562cd016c38f2f833f271e3e99f7b28709136cb15" gracePeriod=600 Jul 24 00:12:53.205714 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.205663 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="kube-rbac-proxy-web" containerID="cri-o://0d529c161d2c7ca7630a06ea9be6df565bce973a84742fa2c604d8e2981d175b" gracePeriod=600 Jul 24 00:12:53.205772 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.205730 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="config-reloader" containerID="cri-o://2d943d93d08716a59163953bd01550d77a94ff4fe1c9aaf21e24177278f593b3" gracePeriod=600 Jul 24 00:12:53.205890 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.205822 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="kube-rbac-proxy-thanos" containerID="cri-o://a03ad8f8f0181bcba1c5d473dccb36d0313d9f81c855c267cb3f120f3c740728" gracePeriod=600 Jul 24 00:12:53.205951 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.205791 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="thanos-sidecar" containerID="cri-o://5c0feadf068a4254ad09ad12f3d180236437dd9ac6df4ae2aa724208a5f5aa81" gracePeriod=600 Jul 24 00:12:53.285354 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.285313 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" Jul 24 00:12:53.435707 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:53.435596 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-577b855d68-tp4ph"] Jul 24 00:12:53.439219 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:12:53.439192 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf54ac913_2a49_4d70_adfc_2fa67aadf7ed.slice/crio-61702bdcedb10114a75cc809c8b98f7cd5246f260c28b91724e126452264576b WatchSource:0}: Error finding container 61702bdcedb10114a75cc809c8b98f7cd5246f260c28b91724e126452264576b: Status 404 returned error can't find the container with id 61702bdcedb10114a75cc809c8b98f7cd5246f260c28b91724e126452264576b Jul 24 00:12:53.988383 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:53.988335 2560 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac is running failed: container process not found" containerID="3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"] Jul 24 00:12:53.988724 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:53.988697 2560 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac is running failed: container process not found" containerID="3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"] Jul 24 00:12:53.988996 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:53.988972 2560 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac is running failed: container process not found" containerID="3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"] Jul 24 00:12:53.989040 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:53.989008 2560 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac is running failed: container process not found" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="prometheus" probeResult="unknown" Jul 24 00:12:54.089569 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.089535 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" event={"ID":"f54ac913-2a49-4d70-adfc-2fa67aadf7ed","Type":"ContainerStarted","Data":"61702bdcedb10114a75cc809c8b98f7cd5246f260c28b91724e126452264576b"} Jul 24 00:12:54.092980 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.092959 2560 generic.go:356] "Generic (PLEG): container finished" podID="2f0a1517-451f-41a0-91b8-382111995fff" containerID="a03ad8f8f0181bcba1c5d473dccb36d0313d9f81c855c267cb3f120f3c740728" exitCode=0 Jul 24 00:12:54.092980 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.092981 2560 generic.go:356] "Generic (PLEG): container finished" podID="2f0a1517-451f-41a0-91b8-382111995fff" containerID="14f702400b95a3bc37f27bb562cd016c38f2f833f271e3e99f7b28709136cb15" exitCode=0 Jul 24 00:12:54.093078 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.092989 2560 generic.go:356] "Generic (PLEG): container finished" podID="2f0a1517-451f-41a0-91b8-382111995fff" containerID="5c0feadf068a4254ad09ad12f3d180236437dd9ac6df4ae2aa724208a5f5aa81" exitCode=0 Jul 24 00:12:54.093078 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.092995 2560 generic.go:356] "Generic (PLEG): container finished" podID="2f0a1517-451f-41a0-91b8-382111995fff" containerID="2d943d93d08716a59163953bd01550d77a94ff4fe1c9aaf21e24177278f593b3" exitCode=0 Jul 24 00:12:54.093078 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.093001 2560 generic.go:356] "Generic (PLEG): container finished" podID="2f0a1517-451f-41a0-91b8-382111995fff" containerID="3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac" exitCode=0 Jul 24 00:12:54.093078 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.093022 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f0a1517-451f-41a0-91b8-382111995fff","Type":"ContainerDied","Data":"a03ad8f8f0181bcba1c5d473dccb36d0313d9f81c855c267cb3f120f3c740728"} Jul 24 00:12:54.093078 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.093069 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f0a1517-451f-41a0-91b8-382111995fff","Type":"ContainerDied","Data":"14f702400b95a3bc37f27bb562cd016c38f2f833f271e3e99f7b28709136cb15"} Jul 24 00:12:54.093222 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.093094 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f0a1517-451f-41a0-91b8-382111995fff","Type":"ContainerDied","Data":"5c0feadf068a4254ad09ad12f3d180236437dd9ac6df4ae2aa724208a5f5aa81"} Jul 24 00:12:54.093222 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.093117 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f0a1517-451f-41a0-91b8-382111995fff","Type":"ContainerDied","Data":"2d943d93d08716a59163953bd01550d77a94ff4fe1c9aaf21e24177278f593b3"} Jul 24 00:12:54.093222 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.093140 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f0a1517-451f-41a0-91b8-382111995fff","Type":"ContainerDied","Data":"3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac"} Jul 24 00:12:54.489461 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.489431 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:54.599490 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599395 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-serving-certs-ca-bundle\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.599490 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599450 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f0a1517-451f-41a0-91b8-382111995fff-config-out\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.599490 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599482 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-web-config\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.599863 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599521 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-db\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.599863 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599584 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-trusted-ca-bundle\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.599863 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599616 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.599863 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599660 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-thanos-prometheus-http-client-file\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.599863 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599694 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f0a1517-451f-41a0-91b8-382111995fff-tls-assets\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.599863 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599732 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-0\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.599863 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599772 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.599863 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599799 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-1\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.599863 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599836 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-kube-rbac-proxy\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.599863 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599861 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-metrics-client-certs\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.600359 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599894 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-2\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.600359 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599908 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:12:54.600359 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599925 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw9sm\" (UniqueName: \"kubernetes.io/projected/2f0a1517-451f-41a0-91b8-382111995fff-kube-api-access-jw9sm\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.600359 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.599990 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-metrics-client-ca\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.600359 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.600053 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-tls\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.600359 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.600168 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-config\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.600359 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.600206 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-grpc-tls\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.600359 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.600250 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-kubelet-serving-ca-bundle\") pod \"2f0a1517-451f-41a0-91b8-382111995fff\" (UID: \"2f0a1517-451f-41a0-91b8-382111995fff\") " Jul 24 00:12:54.600802 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.600564 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.601058 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.601030 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:12:54.602574 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.602542 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:12:54.602809 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.602783 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:54.603791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.602921 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f0a1517-451f-41a0-91b8-382111995fff-config-out" (OuterVolumeSpecName: "config-out") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:12:54.603791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.603443 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:12:54.603791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.603764 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:12:54.604195 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.603822 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:12:54.604195 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.603836 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:12:54.604195 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.603963 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:12:54.604195 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.604047 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:54.604527 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.604293 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-config" (OuterVolumeSpecName: "config") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:54.605275 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.605232 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f0a1517-451f-41a0-91b8-382111995fff-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:12:54.606044 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.606008 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f0a1517-451f-41a0-91b8-382111995fff-kube-api-access-jw9sm" (OuterVolumeSpecName: "kube-api-access-jw9sm") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "kube-api-access-jw9sm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:12:54.606470 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.606410 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:54.606561 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.606505 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:54.606903 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.606866 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:54.607585 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.607557 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:54.607882 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.607843 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:54.618691 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.618654 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-web-config" (OuterVolumeSpecName: "web-config") pod "2f0a1517-451f-41a0-91b8-382111995fff" (UID: "2f0a1517-451f-41a0-91b8-382111995fff"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:12:54.701947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.701909 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.701947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.701943 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw9sm\" (UniqueName: \"kubernetes.io/projected/2f0a1517-451f-41a0-91b8-382111995fff-kube-api-access-jw9sm\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.701961 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-metrics-client-ca\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.701976 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.701990 2560 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-config\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.702005 2560 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-grpc-tls\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.702021 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.702035 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f0a1517-451f-41a0-91b8-382111995fff-config-out\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.702047 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-web-config\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.702061 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-db\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.702073 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.702087 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.702101 2560 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.702116 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f0a1517-451f-41a0-91b8-382111995fff-tls-assets\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.702128 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.702139 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.702153 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f0a1517-451f-41a0-91b8-382111995fff-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.702167 2560 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-kube-rbac-proxy\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:54.702200 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:54.702178 2560 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f0a1517-451f-41a0-91b8-382111995fff-secret-metrics-client-certs\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:12:55.101384 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.101350 2560 generic.go:356] "Generic (PLEG): container finished" podID="2f0a1517-451f-41a0-91b8-382111995fff" containerID="0d529c161d2c7ca7630a06ea9be6df565bce973a84742fa2c604d8e2981d175b" exitCode=0 Jul 24 00:12:55.101847 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.101436 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.101847 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.101436 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f0a1517-451f-41a0-91b8-382111995fff","Type":"ContainerDied","Data":"0d529c161d2c7ca7630a06ea9be6df565bce973a84742fa2c604d8e2981d175b"} Jul 24 00:12:55.101847 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.101520 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f0a1517-451f-41a0-91b8-382111995fff","Type":"ContainerDied","Data":"5afb060c7548ba2b14005124ac754dda40919de7bbcf5452bc23d749151b6a15"} Jul 24 00:12:55.101847 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.101567 2560 scope.go:117] "RemoveContainer" containerID="a03ad8f8f0181bcba1c5d473dccb36d0313d9f81c855c267cb3f120f3c740728" Jul 24 00:12:55.128708 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.128670 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 00:12:55.133304 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.133273 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 00:12:55.160044 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160009 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 00:12:55.160437 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160418 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="prometheus" Jul 24 00:12:55.160518 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160439 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="prometheus" Jul 24 00:12:55.160518 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160456 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="init-config-reloader" Jul 24 00:12:55.160518 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160463 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="init-config-reloader" Jul 24 00:12:55.160518 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160475 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="config-reloader" Jul 24 00:12:55.160518 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160481 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="config-reloader" Jul 24 00:12:55.160518 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160491 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="thanos-sidecar" Jul 24 00:12:55.160518 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160497 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="thanos-sidecar" Jul 24 00:12:55.160518 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160506 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="kube-rbac-proxy-web" Jul 24 00:12:55.160518 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160513 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="kube-rbac-proxy-web" Jul 24 00:12:55.160870 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160530 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="kube-rbac-proxy" Jul 24 00:12:55.160870 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160539 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="kube-rbac-proxy" Jul 24 00:12:55.160870 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160552 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="kube-rbac-proxy-thanos" Jul 24 00:12:55.160870 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160558 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="kube-rbac-proxy-thanos" Jul 24 00:12:55.160870 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160614 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="config-reloader" Jul 24 00:12:55.160870 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160626 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="kube-rbac-proxy-thanos" Jul 24 00:12:55.160870 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160635 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="prometheus" Jul 24 00:12:55.160870 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160655 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="thanos-sidecar" Jul 24 00:12:55.160870 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160666 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="kube-rbac-proxy-web" Jul 24 00:12:55.160870 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.160673 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f0a1517-451f-41a0-91b8-382111995fff" containerName="kube-rbac-proxy" Jul 24 00:12:55.163882 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.163862 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.167264 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.167234 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 24 00:12:55.167391 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.167274 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 24 00:12:55.167391 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.167337 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 24 00:12:55.167516 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.167392 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 24 00:12:55.167516 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.167380 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f49kd2mto0hi\"" Jul 24 00:12:55.167516 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.167244 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 24 00:12:55.167694 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.167619 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 24 00:12:55.167755 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.167716 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 24 00:12:55.167883 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.167865 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 24 00:12:55.168155 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.168138 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 24 00:12:55.168295 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.168277 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jbx9f\"" Jul 24 00:12:55.168347 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.168302 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 24 00:12:55.168402 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.168376 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 24 00:12:55.168628 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.168605 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 24 00:12:55.172898 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.172874 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 24 00:12:55.176851 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.176826 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 24 00:12:55.183159 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.183135 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 00:12:55.308375 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308329 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqvvk\" (UniqueName: \"kubernetes.io/projected/41ebf30d-ef1a-440f-b188-cb96f910a5a4-kube-api-access-tqvvk\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.308570 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308398 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.308570 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308433 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.308570 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308461 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.308570 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308510 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.308803 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308576 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.308803 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308598 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.308803 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308615 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.308803 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308669 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.308803 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308688 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.308803 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308711 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.308803 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308738 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-config\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.308803 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308766 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/41ebf30d-ef1a-440f-b188-cb96f910a5a4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.309209 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308820 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-web-config\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.309209 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308898 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/41ebf30d-ef1a-440f-b188-cb96f910a5a4-config-out\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.309209 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308964 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.309209 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.308991 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/41ebf30d-ef1a-440f-b188-cb96f910a5a4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.309209 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.309018 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.309209 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.309043 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.309209 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.309072 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.409846 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.409748 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.409846 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.409799 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.409846 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.409826 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.409846 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.409848 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.410166 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.409877 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.410166 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.409903 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.410166 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.409926 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.410166 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.409956 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-config\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.410483 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.410445 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.411000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.410970 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.411103 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.411037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/41ebf30d-ef1a-440f-b188-cb96f910a5a4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.411166 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.411099 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.411166 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.411106 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-web-config\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.411166 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.411156 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.411319 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.411165 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/41ebf30d-ef1a-440f-b188-cb96f910a5a4-config-out\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.411319 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.411220 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.411319 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.411248 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/41ebf30d-ef1a-440f-b188-cb96f910a5a4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.411319 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.411279 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.411319 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.411305 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.411550 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.411334 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.411550 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.411367 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqvvk\" (UniqueName: \"kubernetes.io/projected/41ebf30d-ef1a-440f-b188-cb96f910a5a4-kube-api-access-tqvvk\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.411550 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.411410 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.411550 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.411443 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.411550 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.411474 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.413886 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.413090 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.413886 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.413736 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.413886 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.413755 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-config\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.414146 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.414010 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/41ebf30d-ef1a-440f-b188-cb96f910a5a4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.414208 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.414150 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.414208 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.414168 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.414330 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.414288 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.414698 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.414616 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/41ebf30d-ef1a-440f-b188-cb96f910a5a4-config-out\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.414956 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.414915 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.415162 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.415123 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.416509 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.416487 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-web-config\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.416670 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.416633 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.416878 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.416856 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/41ebf30d-ef1a-440f-b188-cb96f910a5a4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.418077 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.418021 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/41ebf30d-ef1a-440f-b188-cb96f910a5a4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.418600 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.418579 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/41ebf30d-ef1a-440f-b188-cb96f910a5a4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.422271 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.422245 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqvvk\" (UniqueName: \"kubernetes.io/projected/41ebf30d-ef1a-440f-b188-cb96f910a5a4-kube-api-access-tqvvk\") pod \"prometheus-k8s-0\" (UID: \"41ebf30d-ef1a-440f-b188-cb96f910a5a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.461351 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.461188 2560 scope.go:117] "RemoveContainer" containerID="14f702400b95a3bc37f27bb562cd016c38f2f833f271e3e99f7b28709136cb15" Jul 24 00:12:55.469022 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.469003 2560 scope.go:117] "RemoveContainer" containerID="0d529c161d2c7ca7630a06ea9be6df565bce973a84742fa2c604d8e2981d175b" Jul 24 00:12:55.482384 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.482353 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:12:55.488852 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.488831 2560 scope.go:117] "RemoveContainer" containerID="5c0feadf068a4254ad09ad12f3d180236437dd9ac6df4ae2aa724208a5f5aa81" Jul 24 00:12:55.497030 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.497005 2560 scope.go:117] "RemoveContainer" containerID="2d943d93d08716a59163953bd01550d77a94ff4fe1c9aaf21e24177278f593b3" Jul 24 00:12:55.513135 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.513112 2560 scope.go:117] "RemoveContainer" containerID="3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac" Jul 24 00:12:55.523442 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.523419 2560 scope.go:117] "RemoveContainer" containerID="755fb24d768c5fa0dc72d61d6120dab6c63cd6c2aa36c9e2e1f434cd37135eb4" Jul 24 00:12:55.541980 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.541902 2560 scope.go:117] "RemoveContainer" containerID="a03ad8f8f0181bcba1c5d473dccb36d0313d9f81c855c267cb3f120f3c740728" Jul 24 00:12:55.543123 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:55.542385 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a03ad8f8f0181bcba1c5d473dccb36d0313d9f81c855c267cb3f120f3c740728\": container with ID starting with a03ad8f8f0181bcba1c5d473dccb36d0313d9f81c855c267cb3f120f3c740728 not found: ID does not exist" containerID="a03ad8f8f0181bcba1c5d473dccb36d0313d9f81c855c267cb3f120f3c740728" Jul 24 00:12:55.543123 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.542422 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a03ad8f8f0181bcba1c5d473dccb36d0313d9f81c855c267cb3f120f3c740728"} err="failed to get container status \"a03ad8f8f0181bcba1c5d473dccb36d0313d9f81c855c267cb3f120f3c740728\": rpc error: code = NotFound desc = could not find container \"a03ad8f8f0181bcba1c5d473dccb36d0313d9f81c855c267cb3f120f3c740728\": container with ID starting with a03ad8f8f0181bcba1c5d473dccb36d0313d9f81c855c267cb3f120f3c740728 not found: ID does not exist" Jul 24 00:12:55.543123 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.542449 2560 scope.go:117] "RemoveContainer" containerID="14f702400b95a3bc37f27bb562cd016c38f2f833f271e3e99f7b28709136cb15" Jul 24 00:12:55.543123 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:55.542802 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14f702400b95a3bc37f27bb562cd016c38f2f833f271e3e99f7b28709136cb15\": container with ID starting with 14f702400b95a3bc37f27bb562cd016c38f2f833f271e3e99f7b28709136cb15 not found: ID does not exist" containerID="14f702400b95a3bc37f27bb562cd016c38f2f833f271e3e99f7b28709136cb15" Jul 24 00:12:55.543123 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.542830 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14f702400b95a3bc37f27bb562cd016c38f2f833f271e3e99f7b28709136cb15"} err="failed to get container status \"14f702400b95a3bc37f27bb562cd016c38f2f833f271e3e99f7b28709136cb15\": rpc error: code = NotFound desc = could not find container \"14f702400b95a3bc37f27bb562cd016c38f2f833f271e3e99f7b28709136cb15\": container with ID starting with 14f702400b95a3bc37f27bb562cd016c38f2f833f271e3e99f7b28709136cb15 not found: ID does not exist" Jul 24 00:12:55.543123 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.542854 2560 scope.go:117] "RemoveContainer" containerID="0d529c161d2c7ca7630a06ea9be6df565bce973a84742fa2c604d8e2981d175b" Jul 24 00:12:55.543624 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:55.543151 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d529c161d2c7ca7630a06ea9be6df565bce973a84742fa2c604d8e2981d175b\": container with ID starting with 0d529c161d2c7ca7630a06ea9be6df565bce973a84742fa2c604d8e2981d175b not found: ID does not exist" containerID="0d529c161d2c7ca7630a06ea9be6df565bce973a84742fa2c604d8e2981d175b" Jul 24 00:12:55.543624 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.543176 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d529c161d2c7ca7630a06ea9be6df565bce973a84742fa2c604d8e2981d175b"} err="failed to get container status \"0d529c161d2c7ca7630a06ea9be6df565bce973a84742fa2c604d8e2981d175b\": rpc error: code = NotFound desc = could not find container \"0d529c161d2c7ca7630a06ea9be6df565bce973a84742fa2c604d8e2981d175b\": container with ID starting with 0d529c161d2c7ca7630a06ea9be6df565bce973a84742fa2c604d8e2981d175b not found: ID does not exist" Jul 24 00:12:55.543624 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.543196 2560 scope.go:117] "RemoveContainer" containerID="5c0feadf068a4254ad09ad12f3d180236437dd9ac6df4ae2aa724208a5f5aa81" Jul 24 00:12:55.543624 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:55.543546 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c0feadf068a4254ad09ad12f3d180236437dd9ac6df4ae2aa724208a5f5aa81\": container with ID starting with 5c0feadf068a4254ad09ad12f3d180236437dd9ac6df4ae2aa724208a5f5aa81 not found: ID does not exist" containerID="5c0feadf068a4254ad09ad12f3d180236437dd9ac6df4ae2aa724208a5f5aa81" Jul 24 00:12:55.543624 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.543588 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c0feadf068a4254ad09ad12f3d180236437dd9ac6df4ae2aa724208a5f5aa81"} err="failed to get container status \"5c0feadf068a4254ad09ad12f3d180236437dd9ac6df4ae2aa724208a5f5aa81\": rpc error: code = NotFound desc = could not find container \"5c0feadf068a4254ad09ad12f3d180236437dd9ac6df4ae2aa724208a5f5aa81\": container with ID starting with 5c0feadf068a4254ad09ad12f3d180236437dd9ac6df4ae2aa724208a5f5aa81 not found: ID does not exist" Jul 24 00:12:55.543624 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.543610 2560 scope.go:117] "RemoveContainer" containerID="2d943d93d08716a59163953bd01550d77a94ff4fe1c9aaf21e24177278f593b3" Jul 24 00:12:55.544131 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:55.544108 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d943d93d08716a59163953bd01550d77a94ff4fe1c9aaf21e24177278f593b3\": container with ID starting with 2d943d93d08716a59163953bd01550d77a94ff4fe1c9aaf21e24177278f593b3 not found: ID does not exist" containerID="2d943d93d08716a59163953bd01550d77a94ff4fe1c9aaf21e24177278f593b3" Jul 24 00:12:55.544215 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.544136 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d943d93d08716a59163953bd01550d77a94ff4fe1c9aaf21e24177278f593b3"} err="failed to get container status \"2d943d93d08716a59163953bd01550d77a94ff4fe1c9aaf21e24177278f593b3\": rpc error: code = NotFound desc = could not find container \"2d943d93d08716a59163953bd01550d77a94ff4fe1c9aaf21e24177278f593b3\": container with ID starting with 2d943d93d08716a59163953bd01550d77a94ff4fe1c9aaf21e24177278f593b3 not found: ID does not exist" Jul 24 00:12:55.544215 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.544149 2560 scope.go:117] "RemoveContainer" containerID="3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac" Jul 24 00:12:55.544361 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:55.544340 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac\": container with ID starting with 3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac not found: ID does not exist" containerID="3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac" Jul 24 00:12:55.544434 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.544359 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac"} err="failed to get container status \"3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac\": rpc error: code = NotFound desc = could not find container \"3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac\": container with ID starting with 3cf524731d1dc4c37ee0d03ae9570280e83435619c4fe4768842eb0e21753dac not found: ID does not exist" Jul 24 00:12:55.544434 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.544372 2560 scope.go:117] "RemoveContainer" containerID="755fb24d768c5fa0dc72d61d6120dab6c63cd6c2aa36c9e2e1f434cd37135eb4" Jul 24 00:12:55.544585 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:12:55.544561 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"755fb24d768c5fa0dc72d61d6120dab6c63cd6c2aa36c9e2e1f434cd37135eb4\": container with ID starting with 755fb24d768c5fa0dc72d61d6120dab6c63cd6c2aa36c9e2e1f434cd37135eb4 not found: ID does not exist" containerID="755fb24d768c5fa0dc72d61d6120dab6c63cd6c2aa36c9e2e1f434cd37135eb4" Jul 24 00:12:55.544585 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.544577 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"755fb24d768c5fa0dc72d61d6120dab6c63cd6c2aa36c9e2e1f434cd37135eb4"} err="failed to get container status \"755fb24d768c5fa0dc72d61d6120dab6c63cd6c2aa36c9e2e1f434cd37135eb4\": rpc error: code = NotFound desc = could not find container \"755fb24d768c5fa0dc72d61d6120dab6c63cd6c2aa36c9e2e1f434cd37135eb4\": container with ID starting with 755fb24d768c5fa0dc72d61d6120dab6c63cd6c2aa36c9e2e1f434cd37135eb4 not found: ID does not exist" Jul 24 00:12:55.640746 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:55.640717 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 00:12:55.644507 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:12:55.644473 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ebf30d_ef1a_440f_b188_cb96f910a5a4.slice/crio-2c5188f68c755142fd7d67dc1aac8cbe522e14f21cf69883a5068082f9a0fb20 WatchSource:0}: Error finding container 2c5188f68c755142fd7d67dc1aac8cbe522e14f21cf69883a5068082f9a0fb20: Status 404 returned error can't find the container with id 2c5188f68c755142fd7d67dc1aac8cbe522e14f21cf69883a5068082f9a0fb20 Jul 24 00:12:56.107586 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:56.107553 2560 generic.go:356] "Generic (PLEG): container finished" podID="41ebf30d-ef1a-440f-b188-cb96f910a5a4" containerID="b8c4a7868379a715302efa5a378ffafacb5ca25ba94a7b568c4f50c31df3ab80" exitCode=0 Jul 24 00:12:56.108016 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:56.107656 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41ebf30d-ef1a-440f-b188-cb96f910a5a4","Type":"ContainerDied","Data":"b8c4a7868379a715302efa5a378ffafacb5ca25ba94a7b568c4f50c31df3ab80"} Jul 24 00:12:56.108016 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:56.107725 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41ebf30d-ef1a-440f-b188-cb96f910a5a4","Type":"ContainerStarted","Data":"2c5188f68c755142fd7d67dc1aac8cbe522e14f21cf69883a5068082f9a0fb20"} Jul 24 00:12:56.109991 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:56.109965 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" event={"ID":"f54ac913-2a49-4d70-adfc-2fa67aadf7ed","Type":"ContainerStarted","Data":"116574e51a673d63044d733a93e289483b3b4a699079e49dcab3f3a9e60d73a7"} Jul 24 00:12:56.110105 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:56.110017 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" event={"ID":"f54ac913-2a49-4d70-adfc-2fa67aadf7ed","Type":"ContainerStarted","Data":"a8ae4c393c5f835734e9a10dabb69478416e8aae861401a6eefa4c0375ee7ebf"} Jul 24 00:12:56.110105 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:56.110042 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" event={"ID":"f54ac913-2a49-4d70-adfc-2fa67aadf7ed","Type":"ContainerStarted","Data":"a07b1ee8b43b777fbef9ffc46687ef6ed6dd8cac841b944c66c7d984f822a55c"} Jul 24 00:12:56.160795 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:56.160736 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-577b855d68-tp4ph" podStartSLOduration=2.103997875 podStartE2EDuration="4.160717501s" totalImagesPullingTime="2.056719626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:12:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:56.159519273 +0000 UTC m=+104.764833590" watchObservedRunningTime="2026-07-24 00:12:56.160717501 +0000 UTC m=+104.766031817" Jul 24 00:12:56.348696 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:56.348664 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f0a1517-451f-41a0-91b8-382111995fff" path="/var/lib/kubelet/pods/2f0a1517-451f-41a0-91b8-382111995fff/volumes" Jul 24 00:12:56.865355 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:56.865322 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6984f4d597-j5vvb"] Jul 24 00:12:56.869602 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:56.869584 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:56.880618 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:56.880595 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6984f4d597-j5vvb"] Jul 24 00:12:56.918487 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:56.918453 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:57.017391 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.017357 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:12:57.026807 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.026775 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-trusted-ca-bundle\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.026969 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.026816 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-oauth-serving-cert\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.026969 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.026925 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-service-ca\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.026969 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.026960 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/504c15d8-446a-4626-83f5-d34d69472d3a-console-serving-cert\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.027072 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.026978 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-console-config\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.027106 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.027069 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbr6b\" (UniqueName: \"kubernetes.io/projected/504c15d8-446a-4626-83f5-d34d69472d3a-kube-api-access-mbr6b\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.027137 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.027124 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/504c15d8-446a-4626-83f5-d34d69472d3a-console-oauth-config\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.122385 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.122276 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41ebf30d-ef1a-440f-b188-cb96f910a5a4","Type":"ContainerStarted","Data":"cf0cde6e5fccb0b1659d432a482a7e5d9a6a280dc78aadda86b85aab658d61c4"} Jul 24 00:12:57.122385 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.122365 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41ebf30d-ef1a-440f-b188-cb96f910a5a4","Type":"ContainerStarted","Data":"6685a928a0d76c8ef115cd3faba4a35312ee2b37ebec3578549ff8540eb13d40"} Jul 24 00:12:57.122825 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.122391 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41ebf30d-ef1a-440f-b188-cb96f910a5a4","Type":"ContainerStarted","Data":"61be11c096371caf9006ddf688c9eb33a92aa339058a5ccede2bbf4d2407b610"} Jul 24 00:12:57.122825 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.122417 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41ebf30d-ef1a-440f-b188-cb96f910a5a4","Type":"ContainerStarted","Data":"0d1005ca31d7a17abbfddfa48aca70449d135720894a55e7a99fdf2de9587922"} Jul 24 00:12:57.122825 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.122443 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41ebf30d-ef1a-440f-b188-cb96f910a5a4","Type":"ContainerStarted","Data":"fc7ef9e21a456e25372a975b83237131820cb58d5819e7bf87e58da36c5ed163"} Jul 24 00:12:57.122825 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.122465 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"41ebf30d-ef1a-440f-b188-cb96f910a5a4","Type":"ContainerStarted","Data":"4ead74647f50d0f48f57fe6aa57509aaa5e8fdf14fcdff75eb11ef8f70e72312"} Jul 24 00:12:57.128227 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.128198 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/504c15d8-446a-4626-83f5-d34d69472d3a-console-oauth-config\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.128447 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.128426 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-trusted-ca-bundle\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.128584 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.128571 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-oauth-serving-cert\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.129368 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.128936 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-service-ca\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.129368 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.128977 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/504c15d8-446a-4626-83f5-d34d69472d3a-console-serving-cert\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.129368 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.129004 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-console-config\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.129368 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.129048 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbr6b\" (UniqueName: \"kubernetes.io/projected/504c15d8-446a-4626-83f5-d34d69472d3a-kube-api-access-mbr6b\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.129368 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.129313 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-oauth-serving-cert\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.129610 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.129375 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-trusted-ca-bundle\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.129769 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.129742 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-console-config\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.129967 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.129948 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-service-ca\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.131348 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.131324 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/504c15d8-446a-4626-83f5-d34d69472d3a-console-oauth-config\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.131465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.131449 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/504c15d8-446a-4626-83f5-d34d69472d3a-console-serving-cert\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.136740 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.136721 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbr6b\" (UniqueName: \"kubernetes.io/projected/504c15d8-446a-4626-83f5-d34d69472d3a-kube-api-access-mbr6b\") pod \"console-6984f4d597-j5vvb\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.157001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.156940 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.135459233 podStartE2EDuration="2.156917244s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.458011ms" isStatefulPod=false podCreationTimestamp="2026-07-24 00:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:57.154594206 +0000 UTC m=+105.759908524" watchObservedRunningTime="2026-07-24 00:12:57.156917244 +0000 UTC m=+105.762231560" Jul 24 00:12:57.180036 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.179990 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:12:57.352817 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:57.352446 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6984f4d597-j5vvb"] Jul 24 00:12:57.354727 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:12:57.354683 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod504c15d8_446a_4626_83f5_d34d69472d3a.slice/crio-5e8b6ed8a1c75a010c1c230c3d46206b4a02524c86076332f9f6f46870d74701 WatchSource:0}: Error finding container 5e8b6ed8a1c75a010c1c230c3d46206b4a02524c86076332f9f6f46870d74701: Status 404 returned error can't find the container with id 5e8b6ed8a1c75a010c1c230c3d46206b4a02524c86076332f9f6f46870d74701 Jul 24 00:12:58.131476 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:58.131441 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6984f4d597-j5vvb" event={"ID":"504c15d8-446a-4626-83f5-d34d69472d3a","Type":"ContainerStarted","Data":"01026d37d7b528a6fba365b473a6a4d979076970027f0d10b09f9891c9e01d45"} Jul 24 00:12:58.131846 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:58.131500 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6984f4d597-j5vvb" event={"ID":"504c15d8-446a-4626-83f5-d34d69472d3a","Type":"ContainerStarted","Data":"5e8b6ed8a1c75a010c1c230c3d46206b4a02524c86076332f9f6f46870d74701"} Jul 24 00:12:58.152659 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:12:58.152588 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6984f4d597-j5vvb" podStartSLOduration=2.152568593 podStartE2EDuration="2.152568593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:12:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:12:58.148215717 +0000 UTC m=+106.753530033" watchObservedRunningTime="2026-07-24 00:12:58.152568593 +0000 UTC m=+106.757882909" Jul 24 00:13:00.483411 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:00.483377 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:13:02.984576 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:02.984546 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2rr2t" Jul 24 00:13:07.180541 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:07.180506 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:13:07.181136 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:07.180568 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:13:07.185391 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:07.185368 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:13:08.185181 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:08.185150 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:13:08.236893 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:08.236862 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-846dc5d995-bhk8z"] Jul 24 00:13:08.345970 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:08.345939 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:13:08.446215 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:08.446131 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:13:08.523574 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:08.523542 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:13:33.258857 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.258801 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-846dc5d995-bhk8z" podUID="0390497f-31b8-4148-a8db-5436b4d7007e" containerName="console" containerID="cri-o://ddbcb8daff9506b8dc976bb3e3322b8ea3edefd091e15db615ca90d58e3b2943" gracePeriod=15 Jul 24 00:13:33.514022 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.513957 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:13:33.670951 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.670910 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-oauth-serving-cert\") pod \"0390497f-31b8-4148-a8db-5436b4d7007e\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " Jul 24 00:13:33.671145 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.670960 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-console-config\") pod \"0390497f-31b8-4148-a8db-5436b4d7007e\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " Jul 24 00:13:33.671145 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.670980 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fv8x4\" (UniqueName: \"kubernetes.io/projected/0390497f-31b8-4148-a8db-5436b4d7007e-kube-api-access-fv8x4\") pod \"0390497f-31b8-4148-a8db-5436b4d7007e\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " Jul 24 00:13:33.671145 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.671031 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-trusted-ca-bundle\") pod \"0390497f-31b8-4148-a8db-5436b4d7007e\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " Jul 24 00:13:33.671145 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.671060 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-service-ca\") pod \"0390497f-31b8-4148-a8db-5436b4d7007e\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " Jul 24 00:13:33.671145 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.671098 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0390497f-31b8-4148-a8db-5436b4d7007e-console-serving-cert\") pod \"0390497f-31b8-4148-a8db-5436b4d7007e\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " Jul 24 00:13:33.671145 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.671122 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0390497f-31b8-4148-a8db-5436b4d7007e-console-oauth-config\") pod \"0390497f-31b8-4148-a8db-5436b4d7007e\" (UID: \"0390497f-31b8-4148-a8db-5436b4d7007e\") " Jul 24 00:13:33.671679 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.671620 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0390497f-31b8-4148-a8db-5436b4d7007e" (UID: "0390497f-31b8-4148-a8db-5436b4d7007e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:13:33.671814 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.671782 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-console-config" (OuterVolumeSpecName: "console-config") pod "0390497f-31b8-4148-a8db-5436b4d7007e" (UID: "0390497f-31b8-4148-a8db-5436b4d7007e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:13:33.672097 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.671934 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0390497f-31b8-4148-a8db-5436b4d7007e" (UID: "0390497f-31b8-4148-a8db-5436b4d7007e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:13:33.672097 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.672047 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-service-ca" (OuterVolumeSpecName: "service-ca") pod "0390497f-31b8-4148-a8db-5436b4d7007e" (UID: "0390497f-31b8-4148-a8db-5436b4d7007e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:13:33.672396 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.672377 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-oauth-serving-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:13:33.672545 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.672523 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-console-config\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:13:33.672545 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.672546 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-trusted-ca-bundle\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:13:33.672740 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.672559 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0390497f-31b8-4148-a8db-5436b4d7007e-service-ca\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:13:33.676687 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.674329 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0390497f-31b8-4148-a8db-5436b4d7007e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0390497f-31b8-4148-a8db-5436b4d7007e" (UID: "0390497f-31b8-4148-a8db-5436b4d7007e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:13:33.676687 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.674672 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0390497f-31b8-4148-a8db-5436b4d7007e-kube-api-access-fv8x4" (OuterVolumeSpecName: "kube-api-access-fv8x4") pod "0390497f-31b8-4148-a8db-5436b4d7007e" (UID: "0390497f-31b8-4148-a8db-5436b4d7007e"). InnerVolumeSpecName "kube-api-access-fv8x4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:13:33.678290 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.678254 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0390497f-31b8-4148-a8db-5436b4d7007e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0390497f-31b8-4148-a8db-5436b4d7007e" (UID: "0390497f-31b8-4148-a8db-5436b4d7007e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:13:33.773176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.773080 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0390497f-31b8-4148-a8db-5436b4d7007e-console-serving-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:13:33.773176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.773121 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0390497f-31b8-4148-a8db-5436b4d7007e-console-oauth-config\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:13:33.773176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:33.773154 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fv8x4\" (UniqueName: \"kubernetes.io/projected/0390497f-31b8-4148-a8db-5436b4d7007e-kube-api-access-fv8x4\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:13:34.288295 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:34.288254 2560 generic.go:356] "Generic (PLEG): container finished" podID="0390497f-31b8-4148-a8db-5436b4d7007e" containerID="ddbcb8daff9506b8dc976bb3e3322b8ea3edefd091e15db615ca90d58e3b2943" exitCode=0 Jul 24 00:13:34.288706 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:34.288318 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846dc5d995-bhk8z" Jul 24 00:13:34.288706 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:34.288331 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846dc5d995-bhk8z" event={"ID":"0390497f-31b8-4148-a8db-5436b4d7007e","Type":"ContainerDied","Data":"ddbcb8daff9506b8dc976bb3e3322b8ea3edefd091e15db615ca90d58e3b2943"} Jul 24 00:13:34.288706 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:34.288408 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846dc5d995-bhk8z" event={"ID":"0390497f-31b8-4148-a8db-5436b4d7007e","Type":"ContainerDied","Data":"ffdd1d95ca06d394cec89a22ad9f8a1fff49a0c16056514ae1262df38a9dd5f7"} Jul 24 00:13:34.288706 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:34.288428 2560 scope.go:117] "RemoveContainer" containerID="ddbcb8daff9506b8dc976bb3e3322b8ea3edefd091e15db615ca90d58e3b2943" Jul 24 00:13:34.297225 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:34.297209 2560 scope.go:117] "RemoveContainer" containerID="ddbcb8daff9506b8dc976bb3e3322b8ea3edefd091e15db615ca90d58e3b2943" Jul 24 00:13:34.297491 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:13:34.297472 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddbcb8daff9506b8dc976bb3e3322b8ea3edefd091e15db615ca90d58e3b2943\": container with ID starting with ddbcb8daff9506b8dc976bb3e3322b8ea3edefd091e15db615ca90d58e3b2943 not found: ID does not exist" containerID="ddbcb8daff9506b8dc976bb3e3322b8ea3edefd091e15db615ca90d58e3b2943" Jul 24 00:13:34.297553 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:34.297502 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddbcb8daff9506b8dc976bb3e3322b8ea3edefd091e15db615ca90d58e3b2943"} err="failed to get container status \"ddbcb8daff9506b8dc976bb3e3322b8ea3edefd091e15db615ca90d58e3b2943\": rpc error: code = NotFound desc = could not find container \"ddbcb8daff9506b8dc976bb3e3322b8ea3edefd091e15db615ca90d58e3b2943\": container with ID starting with ddbcb8daff9506b8dc976bb3e3322b8ea3edefd091e15db615ca90d58e3b2943 not found: ID does not exist" Jul 24 00:13:34.309583 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:34.309559 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-846dc5d995-bhk8z"] Jul 24 00:13:34.313623 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:34.313601 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-846dc5d995-bhk8z"] Jul 24 00:13:34.347006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:34.346979 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0390497f-31b8-4148-a8db-5436b4d7007e" path="/var/lib/kubelet/pods/0390497f-31b8-4148-a8db-5436b4d7007e/volumes" Jul 24 00:13:55.483435 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:55.483388 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:13:55.501571 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:55.501538 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:13:56.400131 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:13:56.400102 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 00:14:02.052931 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:02.052896 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:14:02.112415 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:02.112381 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:14:02.637538 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:02.637497 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:14:03.036356 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:03.036333 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:14:03.093574 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:03.093509 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:14:15.848144 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:15.848108 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:14:15.906437 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:15.906401 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:14:24.700453 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:24.700413 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:14:24.801896 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:24.801862 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:14:46.044147 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.044110 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78fd8fb46b-vq45r"] Jul 24 00:14:46.044577 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.044502 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0390497f-31b8-4148-a8db-5436b4d7007e" containerName="console" Jul 24 00:14:46.044577 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.044515 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0390497f-31b8-4148-a8db-5436b4d7007e" containerName="console" Jul 24 00:14:46.044671 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.044583 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0390497f-31b8-4148-a8db-5436b4d7007e" containerName="console" Jul 24 00:14:46.046588 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.046568 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.060511 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.060485 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78fd8fb46b-vq45r"] Jul 24 00:14:46.093858 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.093825 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:14:46.124503 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.124476 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6fhq\" (UniqueName: \"kubernetes.io/projected/9e292db9-7cff-4721-bca8-7e3ccee982d3-kube-api-access-s6fhq\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.124665 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.124510 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-config\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.124665 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.124547 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-oauth-config\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.124665 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.124626 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-oauth-serving-cert\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.124784 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.124714 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-service-ca\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.124784 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.124731 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-trusted-ca-bundle\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.124784 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.124750 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-serving-cert\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.195000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.194963 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:14:46.225317 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.225283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-oauth-config\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.225453 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.225333 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-oauth-serving-cert\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.225453 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.225392 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-service-ca\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.225453 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.225415 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-trusted-ca-bundle\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.225453 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.225439 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-serving-cert\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.225631 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.225489 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6fhq\" (UniqueName: \"kubernetes.io/projected/9e292db9-7cff-4721-bca8-7e3ccee982d3-kube-api-access-s6fhq\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.225631 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.225515 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-config\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.226205 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.226179 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-service-ca\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.226319 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.226287 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-config\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.226372 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.226315 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-oauth-serving-cert\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.226372 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.226357 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-trusted-ca-bundle\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.227870 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.227850 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-oauth-config\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.228082 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.228059 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-serving-cert\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.233469 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.233453 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6fhq\" (UniqueName: \"kubernetes.io/projected/9e292db9-7cff-4721-bca8-7e3ccee982d3-kube-api-access-s6fhq\") pod \"console-78fd8fb46b-vq45r\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.361002 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.360921 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:46.487142 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.487118 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78fd8fb46b-vq45r"] Jul 24 00:14:46.489699 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:14:46.489671 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e292db9_7cff_4721_bca8_7e3ccee982d3.slice/crio-b3491a86770782575fe3fed33a03cb53714d21fd0c9fb894aff3080bbc89cfea WatchSource:0}: Error finding container b3491a86770782575fe3fed33a03cb53714d21fd0c9fb894aff3080bbc89cfea: Status 404 returned error can't find the container with id b3491a86770782575fe3fed33a03cb53714d21fd0c9fb894aff3080bbc89cfea Jul 24 00:14:46.587982 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.587946 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fd8fb46b-vq45r" event={"ID":"9e292db9-7cff-4721-bca8-7e3ccee982d3","Type":"ContainerStarted","Data":"c5b90469c758ab0bb2a0c22d1997884fe5218045aeed8854290e61a5dd053a64"} Jul 24 00:14:46.588136 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.588005 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fd8fb46b-vq45r" event={"ID":"9e292db9-7cff-4721-bca8-7e3ccee982d3","Type":"ContainerStarted","Data":"b3491a86770782575fe3fed33a03cb53714d21fd0c9fb894aff3080bbc89cfea"} Jul 24 00:14:46.608448 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:46.608398 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78fd8fb46b-vq45r" podStartSLOduration=0.608380265 podStartE2EDuration="608.380265ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:14:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:14:46.60756771 +0000 UTC m=+215.212882027" watchObservedRunningTime="2026-07-24 00:14:46.608380265 +0000 UTC m=+215.213694580" Jul 24 00:14:56.362138 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:56.362100 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:56.362627 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:56.362158 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:56.366821 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:56.366799 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:56.637064 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:56.636984 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:14:56.686844 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:56.686799 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6984f4d597-j5vvb"] Jul 24 00:14:56.794172 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:56.794135 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:14:56.850069 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:56.850030 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:14:56.917015 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:14:56.916936 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:11.148389 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:11.148353 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:11.250340 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:11.250308 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:21.709009 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:21.708958 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6984f4d597-j5vvb" podUID="504c15d8-446a-4626-83f5-d34d69472d3a" containerName="console" containerID="cri-o://01026d37d7b528a6fba365b473a6a4d979076970027f0d10b09f9891c9e01d45" gracePeriod=15 Jul 24 00:15:21.954857 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:21.954833 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:15:22.045928 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.045885 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/504c15d8-446a-4626-83f5-d34d69472d3a-console-serving-cert\") pod \"504c15d8-446a-4626-83f5-d34d69472d3a\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " Jul 24 00:15:22.045928 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.045933 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/504c15d8-446a-4626-83f5-d34d69472d3a-console-oauth-config\") pod \"504c15d8-446a-4626-83f5-d34d69472d3a\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " Jul 24 00:15:22.046178 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.045982 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbr6b\" (UniqueName: \"kubernetes.io/projected/504c15d8-446a-4626-83f5-d34d69472d3a-kube-api-access-mbr6b\") pod \"504c15d8-446a-4626-83f5-d34d69472d3a\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " Jul 24 00:15:22.046178 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.046048 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-oauth-serving-cert\") pod \"504c15d8-446a-4626-83f5-d34d69472d3a\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " Jul 24 00:15:22.046178 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.046084 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-trusted-ca-bundle\") pod \"504c15d8-446a-4626-83f5-d34d69472d3a\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " Jul 24 00:15:22.046178 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.046105 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-console-config\") pod \"504c15d8-446a-4626-83f5-d34d69472d3a\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " Jul 24 00:15:22.046178 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.046134 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-service-ca\") pod \"504c15d8-446a-4626-83f5-d34d69472d3a\" (UID: \"504c15d8-446a-4626-83f5-d34d69472d3a\") " Jul 24 00:15:22.046475 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.046453 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-console-config" (OuterVolumeSpecName: "console-config") pod "504c15d8-446a-4626-83f5-d34d69472d3a" (UID: "504c15d8-446a-4626-83f5-d34d69472d3a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:15:22.046520 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.046487 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "504c15d8-446a-4626-83f5-d34d69472d3a" (UID: "504c15d8-446a-4626-83f5-d34d69472d3a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:15:22.046520 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.046506 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "504c15d8-446a-4626-83f5-d34d69472d3a" (UID: "504c15d8-446a-4626-83f5-d34d69472d3a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:15:22.046595 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.046563 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-service-ca" (OuterVolumeSpecName: "service-ca") pod "504c15d8-446a-4626-83f5-d34d69472d3a" (UID: "504c15d8-446a-4626-83f5-d34d69472d3a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:15:22.048491 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.048466 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/504c15d8-446a-4626-83f5-d34d69472d3a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "504c15d8-446a-4626-83f5-d34d69472d3a" (UID: "504c15d8-446a-4626-83f5-d34d69472d3a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:15:22.048592 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.048558 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/504c15d8-446a-4626-83f5-d34d69472d3a-kube-api-access-mbr6b" (OuterVolumeSpecName: "kube-api-access-mbr6b") pod "504c15d8-446a-4626-83f5-d34d69472d3a" (UID: "504c15d8-446a-4626-83f5-d34d69472d3a"). InnerVolumeSpecName "kube-api-access-mbr6b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:15:22.048592 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.048569 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/504c15d8-446a-4626-83f5-d34d69472d3a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "504c15d8-446a-4626-83f5-d34d69472d3a" (UID: "504c15d8-446a-4626-83f5-d34d69472d3a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:15:22.147221 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.147168 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-oauth-serving-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:15:22.147221 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.147213 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-trusted-ca-bundle\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:15:22.147221 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.147223 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-console-config\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:15:22.147221 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.147232 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/504c15d8-446a-4626-83f5-d34d69472d3a-service-ca\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:15:22.147221 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.147240 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/504c15d8-446a-4626-83f5-d34d69472d3a-console-serving-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:15:22.147514 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.147248 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/504c15d8-446a-4626-83f5-d34d69472d3a-console-oauth-config\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:15:22.147514 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.147256 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbr6b\" (UniqueName: \"kubernetes.io/projected/504c15d8-446a-4626-83f5-d34d69472d3a-kube-api-access-mbr6b\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:15:22.734964 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.734880 2560 generic.go:356] "Generic (PLEG): container finished" podID="504c15d8-446a-4626-83f5-d34d69472d3a" containerID="01026d37d7b528a6fba365b473a6a4d979076970027f0d10b09f9891c9e01d45" exitCode=0 Jul 24 00:15:22.734964 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.734929 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6984f4d597-j5vvb" Jul 24 00:15:22.735401 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.734966 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6984f4d597-j5vvb" event={"ID":"504c15d8-446a-4626-83f5-d34d69472d3a","Type":"ContainerDied","Data":"01026d37d7b528a6fba365b473a6a4d979076970027f0d10b09f9891c9e01d45"} Jul 24 00:15:22.735401 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.735018 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6984f4d597-j5vvb" event={"ID":"504c15d8-446a-4626-83f5-d34d69472d3a","Type":"ContainerDied","Data":"5e8b6ed8a1c75a010c1c230c3d46206b4a02524c86076332f9f6f46870d74701"} Jul 24 00:15:22.735401 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.735039 2560 scope.go:117] "RemoveContainer" containerID="01026d37d7b528a6fba365b473a6a4d979076970027f0d10b09f9891c9e01d45" Jul 24 00:15:22.743302 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.743286 2560 scope.go:117] "RemoveContainer" containerID="01026d37d7b528a6fba365b473a6a4d979076970027f0d10b09f9891c9e01d45" Jul 24 00:15:22.743612 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:15:22.743586 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01026d37d7b528a6fba365b473a6a4d979076970027f0d10b09f9891c9e01d45\": container with ID starting with 01026d37d7b528a6fba365b473a6a4d979076970027f0d10b09f9891c9e01d45 not found: ID does not exist" containerID="01026d37d7b528a6fba365b473a6a4d979076970027f0d10b09f9891c9e01d45" Jul 24 00:15:22.743689 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.743617 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01026d37d7b528a6fba365b473a6a4d979076970027f0d10b09f9891c9e01d45"} err="failed to get container status \"01026d37d7b528a6fba365b473a6a4d979076970027f0d10b09f9891c9e01d45\": rpc error: code = NotFound desc = could not find container \"01026d37d7b528a6fba365b473a6a4d979076970027f0d10b09f9891c9e01d45\": container with ID starting with 01026d37d7b528a6fba365b473a6a4d979076970027f0d10b09f9891c9e01d45 not found: ID does not exist" Jul 24 00:15:22.753222 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.753194 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6984f4d597-j5vvb"] Jul 24 00:15:22.756279 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:22.756254 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6984f4d597-j5vvb"] Jul 24 00:15:24.347162 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:24.347120 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="504c15d8-446a-4626-83f5-d34d69472d3a" path="/var/lib/kubelet/pods/504c15d8-446a-4626-83f5-d34d69472d3a/volumes" Jul 24 00:15:33.023058 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:33.023021 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:34.758197 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:34.758159 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:36.600388 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:36.600355 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:36.667224 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:36.667191 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:37.694354 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:37.694312 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:37.825205 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:37.825158 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:37.981820 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:37.981744 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:40.183903 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:40.183868 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:40.388993 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:40.388958 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:43.404617 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:43.404586 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:43.468209 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:43.468178 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:44.357354 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.357326 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xqxmh"] Jul 24 00:15:44.357800 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.357772 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="504c15d8-446a-4626-83f5-d34d69472d3a" containerName="console" Jul 24 00:15:44.357800 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.357795 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="504c15d8-446a-4626-83f5-d34d69472d3a" containerName="console" Jul 24 00:15:44.357957 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.357896 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="504c15d8-446a-4626-83f5-d34d69472d3a" containerName="console" Jul 24 00:15:44.360744 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.360728 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xqxmh" Jul 24 00:15:44.363490 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.363471 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 24 00:15:44.364522 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.364499 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 24 00:15:44.364603 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.364499 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-dfqrb\"" Jul 24 00:15:44.370264 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.370240 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xqxmh"] Jul 24 00:15:44.448380 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.448334 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkmcj\" (UniqueName: \"kubernetes.io/projected/254295e4-7d91-4f09-bb52-d220d18fbd3d-kube-api-access-lkmcj\") pod \"cert-manager-webhook-6486ffd997-xqxmh\" (UID: \"254295e4-7d91-4f09-bb52-d220d18fbd3d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xqxmh" Jul 24 00:15:44.448786 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.448393 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/254295e4-7d91-4f09-bb52-d220d18fbd3d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xqxmh\" (UID: \"254295e4-7d91-4f09-bb52-d220d18fbd3d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xqxmh" Jul 24 00:15:44.549210 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.549169 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkmcj\" (UniqueName: \"kubernetes.io/projected/254295e4-7d91-4f09-bb52-d220d18fbd3d-kube-api-access-lkmcj\") pod \"cert-manager-webhook-6486ffd997-xqxmh\" (UID: \"254295e4-7d91-4f09-bb52-d220d18fbd3d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xqxmh" Jul 24 00:15:44.549410 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.549218 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/254295e4-7d91-4f09-bb52-d220d18fbd3d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xqxmh\" (UID: \"254295e4-7d91-4f09-bb52-d220d18fbd3d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xqxmh" Jul 24 00:15:44.558001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.557960 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/254295e4-7d91-4f09-bb52-d220d18fbd3d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xqxmh\" (UID: \"254295e4-7d91-4f09-bb52-d220d18fbd3d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xqxmh" Jul 24 00:15:44.558172 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.558151 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkmcj\" (UniqueName: \"kubernetes.io/projected/254295e4-7d91-4f09-bb52-d220d18fbd3d-kube-api-access-lkmcj\") pod \"cert-manager-webhook-6486ffd997-xqxmh\" (UID: \"254295e4-7d91-4f09-bb52-d220d18fbd3d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xqxmh" Jul 24 00:15:44.680830 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.680740 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xqxmh" Jul 24 00:15:44.812080 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.812056 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xqxmh"] Jul 24 00:15:44.814196 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:15:44.814173 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod254295e4_7d91_4f09_bb52_d220d18fbd3d.slice/crio-3bafeaa471db0802cd32a4638cd06d9de7e2d04c78d6d4ffe1fb916dd3b41277 WatchSource:0}: Error finding container 3bafeaa471db0802cd32a4638cd06d9de7e2d04c78d6d4ffe1fb916dd3b41277: Status 404 returned error can't find the container with id 3bafeaa471db0802cd32a4638cd06d9de7e2d04c78d6d4ffe1fb916dd3b41277 Jul 24 00:15:44.825616 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:44.825588 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xqxmh" event={"ID":"254295e4-7d91-4f09-bb52-d220d18fbd3d","Type":"ContainerStarted","Data":"3bafeaa471db0802cd32a4638cd06d9de7e2d04c78d6d4ffe1fb916dd3b41277"} Jul 24 00:15:46.815233 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:46.815190 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:46.914607 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:46.914568 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:47.394471 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:47.394415 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:48.224243 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:48.224164 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:48.324552 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:48.324516 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:48.835481 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:48.835446 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tblp4"] Jul 24 00:15:48.838856 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:48.838835 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tblp4" Jul 24 00:15:48.841255 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:48.841230 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 24 00:15:48.842294 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:48.842274 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-d655j\"" Jul 24 00:15:48.842294 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:48.842289 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 24 00:15:48.849306 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:48.849251 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xqxmh" event={"ID":"254295e4-7d91-4f09-bb52-d220d18fbd3d","Type":"ContainerStarted","Data":"e90ec4694d017486be769719a9269be94691a61fdd7270e8bd32cd7bcf7ed176"} Jul 24 00:15:48.849426 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:48.849392 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xqxmh" Jul 24 00:15:48.850485 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:48.850466 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tblp4"] Jul 24 00:15:48.879168 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:48.879110 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-xqxmh" podStartSLOduration=1.7286151140000001 podStartE2EDuration="4.879092467s" totalImagesPullingTime="3.150477353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:15:48.877314648 +0000 UTC m=+277.482628963" watchObservedRunningTime="2026-07-24 00:15:48.879092467 +0000 UTC m=+277.484406782" Jul 24 00:15:48.991853 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:48.991793 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/580b0146-fbb4-4da7-9a4b-ae78662b8e5b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tblp4\" (UID: \"580b0146-fbb4-4da7-9a4b-ae78662b8e5b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tblp4" Jul 24 00:15:48.992032 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:48.991864 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ssd5\" (UniqueName: \"kubernetes.io/projected/580b0146-fbb4-4da7-9a4b-ae78662b8e5b-kube-api-access-5ssd5\") pod \"openshift-lws-operator-fd8ccff4c-tblp4\" (UID: \"580b0146-fbb4-4da7-9a4b-ae78662b8e5b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tblp4" Jul 24 00:15:49.093309 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:49.093222 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/580b0146-fbb4-4da7-9a4b-ae78662b8e5b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tblp4\" (UID: \"580b0146-fbb4-4da7-9a4b-ae78662b8e5b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tblp4" Jul 24 00:15:49.093309 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:49.093264 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ssd5\" (UniqueName: \"kubernetes.io/projected/580b0146-fbb4-4da7-9a4b-ae78662b8e5b-kube-api-access-5ssd5\") pod \"openshift-lws-operator-fd8ccff4c-tblp4\" (UID: \"580b0146-fbb4-4da7-9a4b-ae78662b8e5b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tblp4" Jul 24 00:15:49.093628 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:49.093609 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/580b0146-fbb4-4da7-9a4b-ae78662b8e5b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tblp4\" (UID: \"580b0146-fbb4-4da7-9a4b-ae78662b8e5b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tblp4" Jul 24 00:15:49.102831 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:49.102800 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ssd5\" (UniqueName: \"kubernetes.io/projected/580b0146-fbb4-4da7-9a4b-ae78662b8e5b-kube-api-access-5ssd5\") pod \"openshift-lws-operator-fd8ccff4c-tblp4\" (UID: \"580b0146-fbb4-4da7-9a4b-ae78662b8e5b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tblp4" Jul 24 00:15:49.149283 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:49.149245 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tblp4" Jul 24 00:15:49.279584 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:49.279556 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tblp4"] Jul 24 00:15:49.281916 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:15:49.281891 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod580b0146_fbb4_4da7_9a4b_ae78662b8e5b.slice/crio-e4a12aab4d3385a940a64829a4c2952f4360b6d5573f462bcd61d28ad95ff9fd WatchSource:0}: Error finding container e4a12aab4d3385a940a64829a4c2952f4360b6d5573f462bcd61d28ad95ff9fd: Status 404 returned error can't find the container with id e4a12aab4d3385a940a64829a4c2952f4360b6d5573f462bcd61d28ad95ff9fd Jul 24 00:15:49.857332 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:49.857290 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tblp4" event={"ID":"580b0146-fbb4-4da7-9a4b-ae78662b8e5b","Type":"ContainerStarted","Data":"e4a12aab4d3385a940a64829a4c2952f4360b6d5573f462bcd61d28ad95ff9fd"} Jul 24 00:15:51.870511 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:51.870418 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tblp4" event={"ID":"580b0146-fbb4-4da7-9a4b-ae78662b8e5b","Type":"ContainerStarted","Data":"81532b371cec18e6c6c39e25c464857eb92739b3143993573eb34751ba59cbad"} Jul 24 00:15:51.888624 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:51.888572 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tblp4" podStartSLOduration=1.554695282 podStartE2EDuration="3.888555102s" totalImagesPullingTime="2.33385982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:15:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:15:51.887477836 +0000 UTC m=+280.492792154" watchObservedRunningTime="2026-07-24 00:15:51.888555102 +0000 UTC m=+280.493869418" Jul 24 00:15:52.653441 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:52.653393 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:52.753949 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:52.753917 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:15:54.859816 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:54.859776 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xqxmh" Jul 24 00:15:54.972320 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:15:54.972286 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:10.636604 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:10.636571 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:10.731849 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:10.731803 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:11.241447 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.241416 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:11.546267 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.546238 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:11.647875 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.647843 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:11.698296 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.698262 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6"] Jul 24 00:16:11.702002 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.701973 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.704766 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.704742 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 00:16:11.704893 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.704793 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 00:16:11.705270 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.705254 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 00:16:11.705462 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.705449 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-g5wcl\"" Jul 24 00:16:11.706041 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.706026 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 00:16:11.719800 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.719770 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6"] Jul 24 00:16:11.793976 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.793946 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8cb4e2fd-ff73-4aed-a48b-6ec842c744ea-manifests\") pod \"opendatahub-operator-controller-manager-7cdb55f47c-jwbd6\" (UID: \"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.794158 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.794017 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8cb4e2fd-ff73-4aed-a48b-6ec842c744ea-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cdb55f47c-jwbd6\" (UID: \"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.794158 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.794094 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hs8h\" (UniqueName: \"kubernetes.io/projected/8cb4e2fd-ff73-4aed-a48b-6ec842c744ea-kube-api-access-7hs8h\") pod \"opendatahub-operator-controller-manager-7cdb55f47c-jwbd6\" (UID: \"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.794158 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.794121 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8cb4e2fd-ff73-4aed-a48b-6ec842c744ea-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cdb55f47c-jwbd6\" (UID: \"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.794158 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.794157 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cb4e2fd-ff73-4aed-a48b-6ec842c744ea-tmp\") pod \"opendatahub-operator-controller-manager-7cdb55f47c-jwbd6\" (UID: \"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.826881 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.826800 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:11.894762 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.894728 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8cb4e2fd-ff73-4aed-a48b-6ec842c744ea-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cdb55f47c-jwbd6\" (UID: \"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.894957 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.894791 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hs8h\" (UniqueName: \"kubernetes.io/projected/8cb4e2fd-ff73-4aed-a48b-6ec842c744ea-kube-api-access-7hs8h\") pod \"opendatahub-operator-controller-manager-7cdb55f47c-jwbd6\" (UID: \"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.894957 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.894818 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8cb4e2fd-ff73-4aed-a48b-6ec842c744ea-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cdb55f47c-jwbd6\" (UID: \"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.894957 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.894851 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cb4e2fd-ff73-4aed-a48b-6ec842c744ea-tmp\") pod \"opendatahub-operator-controller-manager-7cdb55f47c-jwbd6\" (UID: \"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.894957 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.894883 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8cb4e2fd-ff73-4aed-a48b-6ec842c744ea-manifests\") pod \"opendatahub-operator-controller-manager-7cdb55f47c-jwbd6\" (UID: \"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.895244 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.895228 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8cb4e2fd-ff73-4aed-a48b-6ec842c744ea-manifests\") pod \"opendatahub-operator-controller-manager-7cdb55f47c-jwbd6\" (UID: \"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.897082 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.897063 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cb4e2fd-ff73-4aed-a48b-6ec842c744ea-tmp\") pod \"opendatahub-operator-controller-manager-7cdb55f47c-jwbd6\" (UID: \"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.897448 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.897433 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 00:16:11.903400 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.903379 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 00:16:11.907481 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.907462 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8cb4e2fd-ff73-4aed-a48b-6ec842c744ea-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cdb55f47c-jwbd6\" (UID: \"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.907578 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.907511 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8cb4e2fd-ff73-4aed-a48b-6ec842c744ea-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cdb55f47c-jwbd6\" (UID: \"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.913870 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.913843 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 00:16:11.924363 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.924335 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hs8h\" (UniqueName: \"kubernetes.io/projected/8cb4e2fd-ff73-4aed-a48b-6ec842c744ea-kube-api-access-7hs8h\") pod \"opendatahub-operator-controller-manager-7cdb55f47c-jwbd6\" (UID: \"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea\") " pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:11.925559 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:11.925537 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:12.014365 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:12.014340 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-g5wcl\"" Jul 24 00:16:12.021866 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:12.021834 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:12.027857 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:12.027837 2560 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 00:16:12.028530 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:12.028509 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:12.134075 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:12.134042 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:12.204833 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:12.204809 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6"] Jul 24 00:16:12.206958 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:16:12.206933 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cb4e2fd_ff73_4aed_a48b_6ec842c744ea.slice/crio-85cc5ddc415e5033cd14d76f481c79037cb8db8f3a954bd1e9d1bbc1fb2a738d WatchSource:0}: Error finding container 85cc5ddc415e5033cd14d76f481c79037cb8db8f3a954bd1e9d1bbc1fb2a738d: Status 404 returned error can't find the container with id 85cc5ddc415e5033cd14d76f481c79037cb8db8f3a954bd1e9d1bbc1fb2a738d Jul 24 00:16:12.248845 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:12.248812 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:12.396000 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:12.395804 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:12.967788 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:12.967756 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" event={"ID":"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea","Type":"ContainerStarted","Data":"85cc5ddc415e5033cd14d76f481c79037cb8db8f3a954bd1e9d1bbc1fb2a738d"} Jul 24 00:16:15.985597 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:15.985558 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" event={"ID":"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea","Type":"ContainerStarted","Data":"75b7d75f3227ea5abd41608db3ae776e3e48b53d4c1b39edeab3bdd73df506cf"} Jul 24 00:16:16.991222 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:16.991183 2560 generic.go:356] "Generic (PLEG): container finished" podID="8cb4e2fd-ff73-4aed-a48b-6ec842c744ea" containerID="75b7d75f3227ea5abd41608db3ae776e3e48b53d4c1b39edeab3bdd73df506cf" exitCode=0 Jul 24 00:16:16.991724 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:16.991263 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" event={"ID":"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea","Type":"ContainerDied","Data":"75b7d75f3227ea5abd41608db3ae776e3e48b53d4c1b39edeab3bdd73df506cf"} Jul 24 00:16:16.991879 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:16.991861 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 00:16:17.317006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:17.316968 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:17.413192 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:17.413163 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:17.997413 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:17.997379 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" event={"ID":"8cb4e2fd-ff73-4aed-a48b-6ec842c744ea","Type":"ContainerStarted","Data":"1e1e177aa6c13bcc031e52ec5023e5a7a80fda3aed3709753cb85e27b963a1e0"} Jul 24 00:16:17.998020 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:17.997514 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:18.025482 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:18.025426 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" podStartSLOduration=2.679138484 podStartE2EDuration="7.025407589s" totalImagesPullingTime="3.266249439s" totalInitContainerRuntime="1.080019666s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:16:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:16:18.023058642 +0000 UTC m=+306.628372974" watchObservedRunningTime="2026-07-24 00:16:18.025407589 +0000 UTC m=+306.630721974" Jul 24 00:16:29.003739 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:29.003703 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7cdb55f47c-jwbd6" Jul 24 00:16:30.654149 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:30.654106 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:30.796866 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:30.796835 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:31.310080 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:31.310051 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:31.563203 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:31.563101 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:31.664877 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:31.664840 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:31.765296 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:31.765260 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:31.867844 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:31.867724 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:32.101762 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:32.101687 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:32.506384 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:32.506351 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:32.606299 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:32.606264 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:32.765821 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:32.765723 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:32.845767 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:32.845733 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:34.042536 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:34.042500 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:38.683980 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:38.683942 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:38.780009 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:38.779973 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:39.087490 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.087452 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:39.095079 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.095051 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk"] Jul 24 00:16:39.097906 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.097878 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:16:39.103199 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.102876 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 24 00:16:39.103199 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.103034 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 24 00:16:39.103199 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.103062 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 24 00:16:39.103199 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.103147 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-prv66\"" Jul 24 00:16:39.113407 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.113375 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk"] Jul 24 00:16:39.172911 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.172874 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc"] Jul 24 00:16:39.175526 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.175507 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.177961 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.177936 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-twg8t\"" Jul 24 00:16:39.178243 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.178227 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 00:16:39.178587 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.178570 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 00:16:39.178587 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.178575 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 00:16:39.194379 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.194351 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc"] Jul 24 00:16:39.254829 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.254797 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9fde78b7-bbde-4051-832b-808c7a7af8e1-metrics-cert\") pod \"lws-controller-manager-5bbf775ccb-k2qtk\" (UID: \"9fde78b7-bbde-4051-832b-808c7a7af8e1\") " pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:16:39.255012 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.254835 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkk54\" (UniqueName: \"kubernetes.io/projected/9fde78b7-bbde-4051-832b-808c7a7af8e1-kube-api-access-zkk54\") pod \"lws-controller-manager-5bbf775ccb-k2qtk\" (UID: \"9fde78b7-bbde-4051-832b-808c7a7af8e1\") " pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:16:39.255012 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.254859 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9fde78b7-bbde-4051-832b-808c7a7af8e1-manager-config\") pod \"lws-controller-manager-5bbf775ccb-k2qtk\" (UID: \"9fde78b7-bbde-4051-832b-808c7a7af8e1\") " pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:16:39.255012 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.254875 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9fde78b7-bbde-4051-832b-808c7a7af8e1-cert\") pod \"lws-controller-manager-5bbf775ccb-k2qtk\" (UID: \"9fde78b7-bbde-4051-832b-808c7a7af8e1\") " pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:16:39.299750 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.299719 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:39.356405 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.356319 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d50a234-1c26-4737-afe6-c88f47b7e388-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.356405 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.356361 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d50a234-1c26-4737-afe6-c88f47b7e388-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.356405 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.356391 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d50a234-1c26-4737-afe6-c88f47b7e388-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.356715 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.356461 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d50a234-1c26-4737-afe6-c88f47b7e388-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.356715 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.356490 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d50a234-1c26-4737-afe6-c88f47b7e388-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.356715 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.356511 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d50a234-1c26-4737-afe6-c88f47b7e388-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.356715 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.356544 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9fde78b7-bbde-4051-832b-808c7a7af8e1-metrics-cert\") pod \"lws-controller-manager-5bbf775ccb-k2qtk\" (UID: \"9fde78b7-bbde-4051-832b-808c7a7af8e1\") " pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:16:39.356715 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.356600 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkk54\" (UniqueName: \"kubernetes.io/projected/9fde78b7-bbde-4051-832b-808c7a7af8e1-kube-api-access-zkk54\") pod \"lws-controller-manager-5bbf775ccb-k2qtk\" (UID: \"9fde78b7-bbde-4051-832b-808c7a7af8e1\") " pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:16:39.356715 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.356664 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d50a234-1c26-4737-afe6-c88f47b7e388-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.356715 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.356696 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d50a234-1c26-4737-afe6-c88f47b7e388-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.357070 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.356727 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9fde78b7-bbde-4051-832b-808c7a7af8e1-manager-config\") pod \"lws-controller-manager-5bbf775ccb-k2qtk\" (UID: \"9fde78b7-bbde-4051-832b-808c7a7af8e1\") " pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:16:39.357070 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.356758 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9fde78b7-bbde-4051-832b-808c7a7af8e1-cert\") pod \"lws-controller-manager-5bbf775ccb-k2qtk\" (UID: \"9fde78b7-bbde-4051-832b-808c7a7af8e1\") " pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:16:39.357070 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.356827 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w2lp\" (UniqueName: \"kubernetes.io/projected/4d50a234-1c26-4737-afe6-c88f47b7e388-kube-api-access-8w2lp\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.357506 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.357483 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9fde78b7-bbde-4051-832b-808c7a7af8e1-manager-config\") pod \"lws-controller-manager-5bbf775ccb-k2qtk\" (UID: \"9fde78b7-bbde-4051-832b-808c7a7af8e1\") " pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:16:39.359128 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.359107 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9fde78b7-bbde-4051-832b-808c7a7af8e1-cert\") pod \"lws-controller-manager-5bbf775ccb-k2qtk\" (UID: \"9fde78b7-bbde-4051-832b-808c7a7af8e1\") " pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:16:39.359249 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.359233 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9fde78b7-bbde-4051-832b-808c7a7af8e1-metrics-cert\") pod \"lws-controller-manager-5bbf775ccb-k2qtk\" (UID: \"9fde78b7-bbde-4051-832b-808c7a7af8e1\") " pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:16:39.371842 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.371817 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkk54\" (UniqueName: \"kubernetes.io/projected/9fde78b7-bbde-4051-832b-808c7a7af8e1-kube-api-access-zkk54\") pod \"lws-controller-manager-5bbf775ccb-k2qtk\" (UID: \"9fde78b7-bbde-4051-832b-808c7a7af8e1\") " pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:16:39.408354 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.408320 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:16:39.458062 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.457790 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d50a234-1c26-4737-afe6-c88f47b7e388-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.458062 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.457850 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d50a234-1c26-4737-afe6-c88f47b7e388-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.458062 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.457879 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d50a234-1c26-4737-afe6-c88f47b7e388-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.458062 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.457935 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d50a234-1c26-4737-afe6-c88f47b7e388-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.458499 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.458393 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d50a234-1c26-4737-afe6-c88f47b7e388-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.458499 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.458445 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d50a234-1c26-4737-afe6-c88f47b7e388-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.458662 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.458509 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w2lp\" (UniqueName: \"kubernetes.io/projected/4d50a234-1c26-4737-afe6-c88f47b7e388-kube-api-access-8w2lp\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.458662 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.458571 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d50a234-1c26-4737-afe6-c88f47b7e388-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.458662 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.458606 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d50a234-1c26-4737-afe6-c88f47b7e388-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.458662 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.458659 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d50a234-1c26-4737-afe6-c88f47b7e388-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.458880 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.458662 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d50a234-1c26-4737-afe6-c88f47b7e388-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.458960 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.458936 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d50a234-1c26-4737-afe6-c88f47b7e388-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.459043 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.458980 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d50a234-1c26-4737-afe6-c88f47b7e388-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.459269 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.459232 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d50a234-1c26-4737-afe6-c88f47b7e388-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.461320 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.461293 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d50a234-1c26-4737-afe6-c88f47b7e388-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.461611 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.461592 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d50a234-1c26-4737-afe6-c88f47b7e388-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.476199 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.476167 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w2lp\" (UniqueName: \"kubernetes.io/projected/4d50a234-1c26-4737-afe6-c88f47b7e388-kube-api-access-8w2lp\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.478558 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.478535 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d50a234-1c26-4737-afe6-c88f47b7e388-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc\" (UID: \"4d50a234-1c26-4737-afe6-c88f47b7e388\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.485562 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.485542 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:39.567695 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.566922 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk"] Jul 24 00:16:39.572582 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:16:39.572555 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fde78b7_bbde_4051_832b_808c7a7af8e1.slice/crio-f5ab47dd0a840c2307ce3745b666bd77969be355676023818f7dcd26bba82375 WatchSource:0}: Error finding container f5ab47dd0a840c2307ce3745b666bd77969be355676023818f7dcd26bba82375: Status 404 returned error can't find the container with id f5ab47dd0a840c2307ce3745b666bd77969be355676023818f7dcd26bba82375 Jul 24 00:16:39.639828 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.639802 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc"] Jul 24 00:16:39.641489 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:16:39.641466 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d50a234_1c26_4737_afe6_c88f47b7e388.slice/crio-d4ad1719eed131a4f7239f2652f96a65687648e0c0cbcad8ac76a9bb89e7128e WatchSource:0}: Error finding container d4ad1719eed131a4f7239f2652f96a65687648e0c0cbcad8ac76a9bb89e7128e: Status 404 returned error can't find the container with id d4ad1719eed131a4f7239f2652f96a65687648e0c0cbcad8ac76a9bb89e7128e Jul 24 00:16:39.769291 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:39.769251 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:40.102939 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:40.102902 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" event={"ID":"9fde78b7-bbde-4051-832b-808c7a7af8e1","Type":"ContainerStarted","Data":"f5ab47dd0a840c2307ce3745b666bd77969be355676023818f7dcd26bba82375"} Jul 24 00:16:40.104046 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:40.104019 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" event={"ID":"4d50a234-1c26-4737-afe6-c88f47b7e388","Type":"ContainerStarted","Data":"d4ad1719eed131a4f7239f2652f96a65687648e0c0cbcad8ac76a9bb89e7128e"} Jul 24 00:16:42.117769 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.117730 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" event={"ID":"9fde78b7-bbde-4051-832b-808c7a7af8e1","Type":"ContainerStarted","Data":"65296bbcb9f5cc1f3fad3c34f1abddd5b5965d585dcb9ef64f3b9118cb3754f9"} Jul 24 00:16:42.118264 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.117822 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:16:42.155058 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.154995 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" podStartSLOduration=1.280464724 podStartE2EDuration="3.154978295s" totalImagesPullingTime="1.874513571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:16:42.154817825 +0000 UTC m=+330.760132143" watchObservedRunningTime="2026-07-24 00:16:42.154978295 +0000 UTC m=+330.760292623" Jul 24 00:16:42.314341 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.314304 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg"] Jul 24 00:16:42.321719 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.321687 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.326788 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.326762 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 24 00:16:42.326941 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.326762 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 24 00:16:42.326941 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.326925 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 24 00:16:42.327277 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.327258 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2jw8x\"" Jul 24 00:16:42.338054 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.338025 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg"] Jul 24 00:16:42.396237 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.396151 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.396237 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.396206 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.396237 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.396234 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.396516 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.396308 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.396516 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.396381 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.396516 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.396409 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j94wf\" (UniqueName: \"kubernetes.io/projected/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-kube-api-access-j94wf\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.396516 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.396433 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.496898 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.496864 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.497093 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.496953 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.497093 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.496984 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j94wf\" (UniqueName: \"kubernetes.io/projected/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-kube-api-access-j94wf\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.497093 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.497012 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.497093 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.497053 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.497093 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.497093 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.497453 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.497302 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.497845 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.497780 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.499577 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.499536 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.500038 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.500014 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.500038 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.500014 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.500220 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.500203 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.509425 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.509396 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.509741 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.509700 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j94wf\" (UniqueName: \"kubernetes.io/projected/fd4f15a3-8fd1-4602-80f6-612ae7d031dd-kube-api-access-j94wf\") pod \"istiod-openshift-gateway-5677ccb49b-wv7kg\" (UID: \"fd4f15a3-8fd1-4602-80f6-612ae7d031dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.632219 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.632187 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:42.720768 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.720707 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:16:42.720908 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.720820 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:16:42.720908 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.720868 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:16:42.865865 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:42.865837 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg"] Jul 24 00:16:42.867162 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:16:42.867139 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd4f15a3_8fd1_4602_80f6_612ae7d031dd.slice/crio-6bad1131c0eded4c50e6cf24b0aca9985a760f6ccc5d99331834e37740f0215c WatchSource:0}: Error finding container 6bad1131c0eded4c50e6cf24b0aca9985a760f6ccc5d99331834e37740f0215c: Status 404 returned error can't find the container with id 6bad1131c0eded4c50e6cf24b0aca9985a760f6ccc5d99331834e37740f0215c Jul 24 00:16:43.123960 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:43.123590 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" event={"ID":"4d50a234-1c26-4737-afe6-c88f47b7e388","Type":"ContainerStarted","Data":"a73c299099be1d205ab2eec9496e8d0ca56287c1d30f7a94090598623e188842"} Jul 24 00:16:43.125067 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:43.125036 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" event={"ID":"fd4f15a3-8fd1-4602-80f6-612ae7d031dd","Type":"ContainerStarted","Data":"6bad1131c0eded4c50e6cf24b0aca9985a760f6ccc5d99331834e37740f0215c"} Jul 24 00:16:43.155467 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:43.155401 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" podStartSLOduration=1.0785943630000001 podStartE2EDuration="4.15537706s" totalImagesPullingTime="3.076782697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:16:43.153234315 +0000 UTC m=+331.758548632" watchObservedRunningTime="2026-07-24 00:16:43.15537706 +0000 UTC m=+331.760691379" Jul 24 00:16:43.486801 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:43.486704 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:43.491093 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:43.491066 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:44.047326 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:44.047290 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:44.131572 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:44.131467 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:44.133063 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:44.133036 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc" Jul 24 00:16:44.255697 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:44.255593 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:44.409609 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:44.409524 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:44.608887 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:44.608855 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:45.427126 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:45.427093 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:45.494461 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:45.494417 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:16:45.494581 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:45.494546 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:16:45.530510 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:45.530478 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:45.636244 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:45.636211 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:16:46.143028 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:46.142991 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" event={"ID":"fd4f15a3-8fd1-4602-80f6-612ae7d031dd","Type":"ContainerStarted","Data":"3d5e3b6c59dd4e6ec66bcd53d2e7796d3005784b098acb38d9e6b67f65b20edb"} Jul 24 00:16:46.143209 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:46.143087 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:46.144940 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:46.144915 2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-wv7kg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 00:16:46.145058 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:46.144962 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" podUID="fd4f15a3-8fd1-4602-80f6-612ae7d031dd" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 00:16:46.170720 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:46.170664 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" podStartSLOduration=1.545906656 podStartE2EDuration="4.170633167s" totalImagesPullingTime="2.624726511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:16:46.168491841 +0000 UTC m=+334.773806158" watchObservedRunningTime="2026-07-24 00:16:46.170633167 +0000 UTC m=+334.775947483" Jul 24 00:16:47.148367 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:47.148336 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wv7kg" Jul 24 00:16:53.127911 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:16:53.127881 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5bbf775ccb-k2qtk" Jul 24 00:17:32.342016 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:17:32.341919 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:17:32.444777 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:17:32.444743 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:17:32.531370 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:17:32.531336 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:17:45.116810 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:17:45.116759 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:17:45.168172 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:17:45.168142 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:17:49.103825 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:17:49.103753 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:00.304092 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:00.304039 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:04.024104 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:04.024022 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:04.119843 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:04.119800 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:04.466739 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:04.466629 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:06.885118 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:06.885086 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:07.149169 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:07.149091 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:08.311718 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:08.311686 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:11.477542 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:11.477512 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:11.876687 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:11.876658 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:12.249917 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:12.249817 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:14.954239 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:14.954206 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:15.126033 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:15.125996 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78fd8fb46b-vq45r"] Jul 24 00:18:15.232933 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:15.232850 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:15.333706 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:15.333673 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:15.433751 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:15.433722 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:16.294332 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:16.294299 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:16.394626 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:16.394595 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:20.065232 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:20.065189 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:20.166890 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:20.166834 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:21.284981 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:21.284945 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:21.834589 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:21.834550 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:23.478158 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:23.478116 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:23.534482 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:23.534450 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:31.890125 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:31.890074 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:31.938616 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:31.938583 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n9vgp"] Jul 24 00:18:31.942793 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:31.942769 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n9vgp" Jul 24 00:18:31.945369 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:31.945333 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 00:18:31.945509 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:31.945476 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 00:18:31.946399 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:31.946383 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hbkhv\"" Jul 24 00:18:31.949840 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:31.949818 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n9vgp"] Jul 24 00:18:31.992251 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:31.992197 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:32.029394 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:32.029363 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhpj8\" (UniqueName: \"kubernetes.io/projected/a0b4d128-f563-4479-a832-b5eee7594d8d-kube-api-access-qhpj8\") pod \"limitador-operator-controller-manager-859d7b78bd-n9vgp\" (UID: \"a0b4d128-f563-4479-a832-b5eee7594d8d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n9vgp" Jul 24 00:18:32.053782 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:32.053743 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:32.130466 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:32.130430 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhpj8\" (UniqueName: \"kubernetes.io/projected/a0b4d128-f563-4479-a832-b5eee7594d8d-kube-api-access-qhpj8\") pod \"limitador-operator-controller-manager-859d7b78bd-n9vgp\" (UID: \"a0b4d128-f563-4479-a832-b5eee7594d8d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n9vgp" Jul 24 00:18:32.143693 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:32.143592 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhpj8\" (UniqueName: \"kubernetes.io/projected/a0b4d128-f563-4479-a832-b5eee7594d8d-kube-api-access-qhpj8\") pod \"limitador-operator-controller-manager-859d7b78bd-n9vgp\" (UID: \"a0b4d128-f563-4479-a832-b5eee7594d8d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n9vgp" Jul 24 00:18:32.255510 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:32.255476 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n9vgp" Jul 24 00:18:32.409255 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:32.409225 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n9vgp"] Jul 24 00:18:32.409839 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:18:32.409816 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0b4d128_f563_4479_a832_b5eee7594d8d.slice/crio-867fc4602bbb6c6cb11f3c0da5da35b7d220254c1fb1be466f3c23a81e314b41 WatchSource:0}: Error finding container 867fc4602bbb6c6cb11f3c0da5da35b7d220254c1fb1be466f3c23a81e314b41: Status 404 returned error can't find the container with id 867fc4602bbb6c6cb11f3c0da5da35b7d220254c1fb1be466f3c23a81e314b41 Jul 24 00:18:32.656950 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:32.656867 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n9vgp" event={"ID":"a0b4d128-f563-4479-a832-b5eee7594d8d","Type":"ContainerStarted","Data":"867fc4602bbb6c6cb11f3c0da5da35b7d220254c1fb1be466f3c23a81e314b41"} Jul 24 00:18:34.423851 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:34.423755 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:34.526221 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:34.526183 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:34.667927 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:34.667890 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n9vgp" event={"ID":"a0b4d128-f563-4479-a832-b5eee7594d8d","Type":"ContainerStarted","Data":"baf9ea214592ed20102b2dc5d0285ea45f0e361c64a1cf023899ae045affb0be"} Jul 24 00:18:34.668118 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:34.667973 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n9vgp" Jul 24 00:18:34.686035 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:34.685930 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n9vgp" podStartSLOduration=2.095014315 podStartE2EDuration="3.685915162s" totalImagesPullingTime="1.590900847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:18:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:18:34.682857284 +0000 UTC m=+443.288171602" watchObservedRunningTime="2026-07-24 00:18:34.685915162 +0000 UTC m=+443.291229559" Jul 24 00:18:37.255602 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:37.255564 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:37.355915 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:37.355884 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:40.145493 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.145445 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78fd8fb46b-vq45r" podUID="9e292db9-7cff-4721-bca8-7e3ccee982d3" containerName="console" containerID="cri-o://c5b90469c758ab0bb2a0c22d1997884fe5218045aeed8854290e61a5dd053a64" gracePeriod=15 Jul 24 00:18:40.403623 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.403563 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:18:40.505799 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.505771 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6fhq\" (UniqueName: \"kubernetes.io/projected/9e292db9-7cff-4721-bca8-7e3ccee982d3-kube-api-access-s6fhq\") pod \"9e292db9-7cff-4721-bca8-7e3ccee982d3\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " Jul 24 00:18:40.505981 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.505824 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-config\") pod \"9e292db9-7cff-4721-bca8-7e3ccee982d3\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " Jul 24 00:18:40.505981 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.505866 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-trusted-ca-bundle\") pod \"9e292db9-7cff-4721-bca8-7e3ccee982d3\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " Jul 24 00:18:40.505981 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.505953 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-oauth-serving-cert\") pod \"9e292db9-7cff-4721-bca8-7e3ccee982d3\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " Jul 24 00:18:40.506122 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.506013 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-oauth-config\") pod \"9e292db9-7cff-4721-bca8-7e3ccee982d3\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " Jul 24 00:18:40.506122 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.506059 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-serving-cert\") pod \"9e292db9-7cff-4721-bca8-7e3ccee982d3\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " Jul 24 00:18:40.506228 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.506121 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-service-ca\") pod \"9e292db9-7cff-4721-bca8-7e3ccee982d3\" (UID: \"9e292db9-7cff-4721-bca8-7e3ccee982d3\") " Jul 24 00:18:40.506469 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.506423 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-config" (OuterVolumeSpecName: "console-config") pod "9e292db9-7cff-4721-bca8-7e3ccee982d3" (UID: "9e292db9-7cff-4721-bca8-7e3ccee982d3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:18:40.506764 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.506554 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9e292db9-7cff-4721-bca8-7e3ccee982d3" (UID: "9e292db9-7cff-4721-bca8-7e3ccee982d3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:18:40.506764 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.506606 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9e292db9-7cff-4721-bca8-7e3ccee982d3" (UID: "9e292db9-7cff-4721-bca8-7e3ccee982d3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:18:40.506979 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.506875 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-config\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:18:40.506979 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.506896 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-trusted-ca-bundle\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:18:40.506979 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.506910 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-oauth-serving-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:18:40.507704 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.507669 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-service-ca" (OuterVolumeSpecName: "service-ca") pod "9e292db9-7cff-4721-bca8-7e3ccee982d3" (UID: "9e292db9-7cff-4721-bca8-7e3ccee982d3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:18:40.508548 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.508492 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e292db9-7cff-4721-bca8-7e3ccee982d3-kube-api-access-s6fhq" (OuterVolumeSpecName: "kube-api-access-s6fhq") pod "9e292db9-7cff-4721-bca8-7e3ccee982d3" (UID: "9e292db9-7cff-4721-bca8-7e3ccee982d3"). InnerVolumeSpecName "kube-api-access-s6fhq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:18:40.508954 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.508922 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9e292db9-7cff-4721-bca8-7e3ccee982d3" (UID: "9e292db9-7cff-4721-bca8-7e3ccee982d3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:18:40.509844 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.509818 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9e292db9-7cff-4721-bca8-7e3ccee982d3" (UID: "9e292db9-7cff-4721-bca8-7e3ccee982d3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:18:40.607538 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.607502 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-oauth-config\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:18:40.607538 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.607533 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e292db9-7cff-4721-bca8-7e3ccee982d3-console-serving-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:18:40.607538 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.607543 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e292db9-7cff-4721-bca8-7e3ccee982d3-service-ca\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:18:40.607813 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.607553 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6fhq\" (UniqueName: \"kubernetes.io/projected/9e292db9-7cff-4721-bca8-7e3ccee982d3-kube-api-access-s6fhq\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:18:40.696942 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.696842 2560 generic.go:356] "Generic (PLEG): container finished" podID="9e292db9-7cff-4721-bca8-7e3ccee982d3" containerID="c5b90469c758ab0bb2a0c22d1997884fe5218045aeed8854290e61a5dd053a64" exitCode=0 Jul 24 00:18:40.696942 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.696907 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fd8fb46b-vq45r" Jul 24 00:18:40.696942 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.696922 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fd8fb46b-vq45r" event={"ID":"9e292db9-7cff-4721-bca8-7e3ccee982d3","Type":"ContainerDied","Data":"c5b90469c758ab0bb2a0c22d1997884fe5218045aeed8854290e61a5dd053a64"} Jul 24 00:18:40.697177 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.697007 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fd8fb46b-vq45r" event={"ID":"9e292db9-7cff-4721-bca8-7e3ccee982d3","Type":"ContainerDied","Data":"b3491a86770782575fe3fed33a03cb53714d21fd0c9fb894aff3080bbc89cfea"} Jul 24 00:18:40.697177 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.697035 2560 scope.go:117] "RemoveContainer" containerID="c5b90469c758ab0bb2a0c22d1997884fe5218045aeed8854290e61a5dd053a64" Jul 24 00:18:40.706217 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.706198 2560 scope.go:117] "RemoveContainer" containerID="c5b90469c758ab0bb2a0c22d1997884fe5218045aeed8854290e61a5dd053a64" Jul 24 00:18:40.706492 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:18:40.706470 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5b90469c758ab0bb2a0c22d1997884fe5218045aeed8854290e61a5dd053a64\": container with ID starting with c5b90469c758ab0bb2a0c22d1997884fe5218045aeed8854290e61a5dd053a64 not found: ID does not exist" containerID="c5b90469c758ab0bb2a0c22d1997884fe5218045aeed8854290e61a5dd053a64" Jul 24 00:18:40.706539 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.706500 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5b90469c758ab0bb2a0c22d1997884fe5218045aeed8854290e61a5dd053a64"} err="failed to get container status \"c5b90469c758ab0bb2a0c22d1997884fe5218045aeed8854290e61a5dd053a64\": rpc error: code = NotFound desc = could not find container \"c5b90469c758ab0bb2a0c22d1997884fe5218045aeed8854290e61a5dd053a64\": container with ID starting with c5b90469c758ab0bb2a0c22d1997884fe5218045aeed8854290e61a5dd053a64 not found: ID does not exist" Jul 24 00:18:40.718628 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.718598 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78fd8fb46b-vq45r"] Jul 24 00:18:40.727064 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.727032 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78fd8fb46b-vq45r"] Jul 24 00:18:40.756754 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.756728 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:40.856956 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:40.856925 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:42.347809 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:42.347779 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e292db9-7cff-4721-bca8-7e3ccee982d3" path="/var/lib/kubelet/pods/9e292db9-7cff-4721-bca8-7e3ccee982d3/volumes" Jul 24 00:18:45.675027 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:45.674999 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n9vgp" Jul 24 00:18:45.831964 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:45.831929 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:45.934775 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:45.934690 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:47.437054 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:47.437019 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:49.600186 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:49.600148 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:49.703148 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:49.703110 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:52.201708 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:52.201669 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:52.301335 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:52.301298 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:54.376380 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:54.376349 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:55.402001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:55.401959 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:55.503244 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:55.503213 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:59.002156 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:59.002115 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:18:59.102671 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:18:59.102621 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:08.006335 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:08.006299 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:08.107260 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:08.107225 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:13.788356 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:13.788312 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:14.028061 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:14.028026 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:25.693351 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:25.693315 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:25.801205 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:25.801171 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:25.900911 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:25.900875 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:26.517809 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:26.517776 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:26.574140 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:26.574104 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:31.080067 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:31.080033 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:31.137703 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:31.137662 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:42.197117 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:42.197080 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:42.323407 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:42.323376 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:42.687823 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:42.687779 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:55.028490 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:55.028451 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:19:55.129055 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:19:55.129022 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:20:01.707219 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:20:01.707180 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:20:01.765621 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:20:01.765586 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:20:01.901634 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:20:01.901601 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:20:02.001005 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:20:02.000970 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:20:05.485734 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:20:05.485625 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:20:05.584559 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:20:05.584527 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:20:18.891942 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:20:18.891906 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:20:18.940889 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:20:18.940858 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:20:20.088784 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:20:20.088753 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:20:38.143854 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:20:38.143814 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:20:39.851853 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:20:39.851818 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:20:49.001425 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:20:49.001395 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:20:49.100205 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:20:49.100165 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:20:59.864858 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:20:59.864827 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:21:20.728520 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.728477 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw"] Jul 24 00:21:20.730950 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.728879 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e292db9-7cff-4721-bca8-7e3ccee982d3" containerName="console" Jul 24 00:21:20.730950 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.728892 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e292db9-7cff-4721-bca8-7e3ccee982d3" containerName="console" Jul 24 00:21:20.730950 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.728954 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e292db9-7cff-4721-bca8-7e3ccee982d3" containerName="console" Jul 24 00:21:20.731864 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.731847 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.734426 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.734407 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 24 00:21:20.735731 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.735711 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 24 00:21:20.735784 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.735739 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 24 00:21:20.735865 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.735847 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k8tlc\"" Jul 24 00:21:20.739675 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.739655 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw"] Jul 24 00:21:20.766251 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.766217 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:21:20.769906 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.769874 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.770048 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.769957 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.770048 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.769995 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.770048 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.770019 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.770048 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.770042 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.770242 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.770073 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckbss\" (UniqueName: \"kubernetes.io/projected/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-kube-api-access-ckbss\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.871138 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.871100 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.871138 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.871140 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.871411 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.871173 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckbss\" (UniqueName: \"kubernetes.io/projected/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-kube-api-access-ckbss\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.871411 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.871257 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.871411 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.871350 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.871411 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.871375 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.871674 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.871621 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.871754 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.871691 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.871808 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.871705 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.873395 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.873372 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.873763 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.873743 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:20.878874 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:20.878852 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckbss\" (UniqueName: \"kubernetes.io/projected/e78a4eac-43c4-455a-a37c-ba3dee8e24eb-kube-api-access-ckbss\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vrvxw\" (UID: \"e78a4eac-43c4-455a-a37c-ba3dee8e24eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:21.043659 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:21.043602 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:21.175505 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:21.175455 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw"] Jul 24 00:21:21.177896 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:21:21.177871 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode78a4eac_43c4_455a_a37c_ba3dee8e24eb.slice/crio-f57704fbeb66c497d9eb469ed624a3519d11225447febe392be867aca6fa4dde WatchSource:0}: Error finding container f57704fbeb66c497d9eb469ed624a3519d11225447febe392be867aca6fa4dde: Status 404 returned error can't find the container with id f57704fbeb66c497d9eb469ed624a3519d11225447febe392be867aca6fa4dde Jul 24 00:21:21.179228 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:21.179209 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 00:21:21.199693 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:21.199664 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:21:21.251144 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:21.251113 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:21:21.459935 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:21.459848 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" event={"ID":"e78a4eac-43c4-455a-a37c-ba3dee8e24eb","Type":"ContainerStarted","Data":"f57704fbeb66c497d9eb469ed624a3519d11225447febe392be867aca6fa4dde"} Jul 24 00:21:22.465704 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:22.465613 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" event={"ID":"e78a4eac-43c4-455a-a37c-ba3dee8e24eb","Type":"ContainerStarted","Data":"65ce580f7c667e0b107272675e10e1a5b5896e2b3e237585d8c60dba80cba1eb"} Jul 24 00:21:22.482951 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:22.482898 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" podStartSLOduration=1.535155607 podStartE2EDuration="2.482884296s" totalImagesPullingTime="947.728689ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:21:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:21:22.480733171 +0000 UTC m=+611.086047489" watchObservedRunningTime="2026-07-24 00:21:22.482884296 +0000 UTC m=+611.088198611" Jul 24 00:21:31.044591 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:31.044536 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:31.045037 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:31.044622 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:31.057484 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:31.057462 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:31.520829 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:31.520797 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vrvxw" Jul 24 00:21:39.524385 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.524302 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7"] Jul 24 00:21:39.528605 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.528578 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.531022 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.531000 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 24 00:21:39.536947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.536596 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7"] Jul 24 00:21:39.559891 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.559859 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:21:39.654598 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.654563 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d800604-60f1-4cda-9aaf-92a4d6a8864c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.654806 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.654682 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d800604-60f1-4cda-9aaf-92a4d6a8864c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.654806 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.654720 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d800604-60f1-4cda-9aaf-92a4d6a8864c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.654806 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.654767 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d800604-60f1-4cda-9aaf-92a4d6a8864c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.654923 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.654809 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d800604-60f1-4cda-9aaf-92a4d6a8864c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.654923 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.654846 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gx7l\" (UniqueName: \"kubernetes.io/projected/8d800604-60f1-4cda-9aaf-92a4d6a8864c-kube-api-access-5gx7l\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.756086 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.756043 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d800604-60f1-4cda-9aaf-92a4d6a8864c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.756086 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.756085 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d800604-60f1-4cda-9aaf-92a4d6a8864c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.756317 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.756213 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d800604-60f1-4cda-9aaf-92a4d6a8864c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.756317 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.756268 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d800604-60f1-4cda-9aaf-92a4d6a8864c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.756413 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.756315 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gx7l\" (UniqueName: \"kubernetes.io/projected/8d800604-60f1-4cda-9aaf-92a4d6a8864c-kube-api-access-5gx7l\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.756413 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.756350 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d800604-60f1-4cda-9aaf-92a4d6a8864c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.756498 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.756440 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d800604-60f1-4cda-9aaf-92a4d6a8864c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.756554 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.756489 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d800604-60f1-4cda-9aaf-92a4d6a8864c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.756554 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.756525 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d800604-60f1-4cda-9aaf-92a4d6a8864c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.758543 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.758518 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d800604-60f1-4cda-9aaf-92a4d6a8864c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.758737 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.758718 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d800604-60f1-4cda-9aaf-92a4d6a8864c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.763736 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.763708 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gx7l\" (UniqueName: \"kubernetes.io/projected/8d800604-60f1-4cda-9aaf-92a4d6a8864c-kube-api-access-5gx7l\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rrpn7\" (UID: \"8d800604-60f1-4cda-9aaf-92a4d6a8864c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.840717 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.840611 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:39.967120 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:39.967094 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7"] Jul 24 00:21:39.969371 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:21:39.969348 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d800604_60f1_4cda_9aaf_92a4d6a8864c.slice/crio-26ae2ffffd90a6a94490ff66916112a847ca34bec7ebeb2bc6e8bedce5f93330 WatchSource:0}: Error finding container 26ae2ffffd90a6a94490ff66916112a847ca34bec7ebeb2bc6e8bedce5f93330: Status 404 returned error can't find the container with id 26ae2ffffd90a6a94490ff66916112a847ca34bec7ebeb2bc6e8bedce5f93330 Jul 24 00:21:40.557851 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:40.557813 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" event={"ID":"8d800604-60f1-4cda-9aaf-92a4d6a8864c","Type":"ContainerStarted","Data":"985e0bd8d5deec23dcac5784ba8e2f0c2529b8de44b3d1cff45c4ce3ba7068d7"} Jul 24 00:21:40.558238 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:40.557870 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" event={"ID":"8d800604-60f1-4cda-9aaf-92a4d6a8864c","Type":"ContainerStarted","Data":"26ae2ffffd90a6a94490ff66916112a847ca34bec7ebeb2bc6e8bedce5f93330"} Jul 24 00:21:40.575936 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:40.575886 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" podStartSLOduration=1.398673097 podStartE2EDuration="1.57586934s" totalImagesPullingTime="177.196243ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:21:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:21:40.573412208 +0000 UTC m=+629.178726526" watchObservedRunningTime="2026-07-24 00:21:40.57586934 +0000 UTC m=+629.181183656" Jul 24 00:21:45.152056 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:45.152021 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:21:47.724148 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:47.724114 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:21:49.841425 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:49.841369 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:49.841879 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:49.841445 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:49.854299 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:49.854272 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:50.616897 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:50.616873 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rrpn7" Jul 24 00:21:54.463774 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:21:54.463741 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:23:23.103611 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:23:23.103522 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:23:23.195251 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:23:23.195214 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:23:24.301984 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:23:24.301956 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:23:24.401013 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:23:24.400978 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:24:10.580274 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:24:10.580245 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:25:13.120274 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:13.120238 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b9bbd68d9-l8pwj"] Jul 24 00:25:13.123737 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:13.123721 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" Jul 24 00:25:13.127388 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:13.127368 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 24 00:25:13.127533 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:13.127404 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 24 00:25:13.127533 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:13.127375 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2njs6\"" Jul 24 00:25:13.131588 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:13.131563 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b9bbd68d9-l8pwj"] Jul 24 00:25:13.250125 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:13.250085 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da-webhook-cert\") pod \"maas-controller-5b9bbd68d9-l8pwj\" (UID: \"e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da\") " pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" Jul 24 00:25:13.250315 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:13.250205 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh76h\" (UniqueName: \"kubernetes.io/projected/e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da-kube-api-access-qh76h\") pod \"maas-controller-5b9bbd68d9-l8pwj\" (UID: \"e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da\") " pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" Jul 24 00:25:13.351486 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:13.351445 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da-webhook-cert\") pod \"maas-controller-5b9bbd68d9-l8pwj\" (UID: \"e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da\") " pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" Jul 24 00:25:13.351681 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:13.351526 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh76h\" (UniqueName: \"kubernetes.io/projected/e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da-kube-api-access-qh76h\") pod \"maas-controller-5b9bbd68d9-l8pwj\" (UID: \"e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da\") " pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" Jul 24 00:25:13.353871 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:13.353846 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da-webhook-cert\") pod \"maas-controller-5b9bbd68d9-l8pwj\" (UID: \"e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da\") " pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" Jul 24 00:25:13.358980 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:13.358958 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh76h\" (UniqueName: \"kubernetes.io/projected/e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da-kube-api-access-qh76h\") pod \"maas-controller-5b9bbd68d9-l8pwj\" (UID: \"e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da\") " pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" Jul 24 00:25:13.435493 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:13.435405 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" Jul 24 00:25:13.571539 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:13.571513 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b9bbd68d9-l8pwj"] Jul 24 00:25:13.574226 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:25:13.574199 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7d0cb83_0fb2_48aa_8fb6_2f1af32c97da.slice/crio-70f27d85b3c2cfcf10cc133dfe697b019424104786a24647a451ab7beb779ac5 WatchSource:0}: Error finding container 70f27d85b3c2cfcf10cc133dfe697b019424104786a24647a451ab7beb779ac5: Status 404 returned error can't find the container with id 70f27d85b3c2cfcf10cc133dfe697b019424104786a24647a451ab7beb779ac5 Jul 24 00:25:13.619133 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:13.619102 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:25:14.578013 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:14.577966 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" event={"ID":"e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da","Type":"ContainerStarted","Data":"70f27d85b3c2cfcf10cc133dfe697b019424104786a24647a451ab7beb779ac5"} Jul 24 00:25:16.589207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:16.589172 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" event={"ID":"e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da","Type":"ContainerStarted","Data":"4124826aa79357c583ff0a53fff03b6b0f0fef742d71b7f621f12a2000e90f9c"} Jul 24 00:25:16.589590 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:16.589238 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" Jul 24 00:25:16.608569 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:16.608521 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" podStartSLOduration=1.168360985 podStartE2EDuration="3.608506495s" totalImagesPullingTime="2.44014551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:25:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:25:16.606368104 +0000 UTC m=+845.211682421" watchObservedRunningTime="2026-07-24 00:25:16.608506495 +0000 UTC m=+845.213820810" Jul 24 00:25:27.597308 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:27.597276 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" Jul 24 00:25:32.162501 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.162469 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:25:32.283266 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.283230 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b9bbd68d9-l8pwj"] Jul 24 00:25:32.283448 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.283427 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" podUID="e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da" containerName="manager" containerID="cri-o://4124826aa79357c583ff0a53fff03b6b0f0fef742d71b7f621f12a2000e90f9c" gracePeriod=30 Jul 24 00:25:32.540858 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.540834 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" Jul 24 00:25:32.633870 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.633839 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da-webhook-cert\") pod \"e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da\" (UID: \"e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da\") " Jul 24 00:25:32.634056 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.633901 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh76h\" (UniqueName: \"kubernetes.io/projected/e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da-kube-api-access-qh76h\") pod \"e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da\" (UID: \"e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da\") " Jul 24 00:25:32.636152 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.636118 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da-kube-api-access-qh76h" (OuterVolumeSpecName: "kube-api-access-qh76h") pod "e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da" (UID: "e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da"). InnerVolumeSpecName "kube-api-access-qh76h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:25:32.636152 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.636134 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da" (UID: "e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 00:25:32.669334 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.669300 2560 generic.go:356] "Generic (PLEG): container finished" podID="e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da" containerID="4124826aa79357c583ff0a53fff03b6b0f0fef742d71b7f621f12a2000e90f9c" exitCode=0 Jul 24 00:25:32.669505 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.669347 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" Jul 24 00:25:32.669505 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.669357 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" event={"ID":"e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da","Type":"ContainerDied","Data":"4124826aa79357c583ff0a53fff03b6b0f0fef742d71b7f621f12a2000e90f9c"} Jul 24 00:25:32.669505 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.669443 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b9bbd68d9-l8pwj" event={"ID":"e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da","Type":"ContainerDied","Data":"70f27d85b3c2cfcf10cc133dfe697b019424104786a24647a451ab7beb779ac5"} Jul 24 00:25:32.669505 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.669472 2560 scope.go:117] "RemoveContainer" containerID="4124826aa79357c583ff0a53fff03b6b0f0fef742d71b7f621f12a2000e90f9c" Jul 24 00:25:32.678136 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.678119 2560 scope.go:117] "RemoveContainer" containerID="4124826aa79357c583ff0a53fff03b6b0f0fef742d71b7f621f12a2000e90f9c" Jul 24 00:25:32.678404 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:25:32.678383 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4124826aa79357c583ff0a53fff03b6b0f0fef742d71b7f621f12a2000e90f9c\": container with ID starting with 4124826aa79357c583ff0a53fff03b6b0f0fef742d71b7f621f12a2000e90f9c not found: ID does not exist" containerID="4124826aa79357c583ff0a53fff03b6b0f0fef742d71b7f621f12a2000e90f9c" Jul 24 00:25:32.678457 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.678413 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4124826aa79357c583ff0a53fff03b6b0f0fef742d71b7f621f12a2000e90f9c"} err="failed to get container status \"4124826aa79357c583ff0a53fff03b6b0f0fef742d71b7f621f12a2000e90f9c\": rpc error: code = NotFound desc = could not find container \"4124826aa79357c583ff0a53fff03b6b0f0fef742d71b7f621f12a2000e90f9c\": container with ID starting with 4124826aa79357c583ff0a53fff03b6b0f0fef742d71b7f621f12a2000e90f9c not found: ID does not exist" Jul 24 00:25:32.689814 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.689790 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b9bbd68d9-l8pwj"] Jul 24 00:25:32.695416 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.695391 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b9bbd68d9-l8pwj"] Jul 24 00:25:32.734841 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.734764 2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da-webhook-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:25:32.734841 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:32.734813 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh76h\" (UniqueName: \"kubernetes.io/projected/e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da-kube-api-access-qh76h\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:25:33.049742 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:33.049713 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:25:33.098804 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:33.098774 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:25:33.396833 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:33.396747 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:25:34.347935 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:25:34.347898 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da" path="/var/lib/kubelet/pods/e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da/volumes" Jul 24 00:26:04.171070 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:26:04.170994 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:26:05.341393 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:26:05.341360 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:26:10.146563 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:26:10.146533 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:26:10.256102 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:26:10.256068 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:26:10.346418 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:26:10.346381 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:26:10.510404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:26:10.510369 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:26:16.001379 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:26:16.001343 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:26:16.064386 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:26:16.064349 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:27:47.291220 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:27:47.291121 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:27:47.377515 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:27:47.377457 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:30:00.139830 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:00.139787 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747550-69wkt"] Jul 24 00:30:00.140447 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:00.140367 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da" containerName="manager" Jul 24 00:30:00.140447 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:00.140388 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da" containerName="manager" Jul 24 00:30:00.140567 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:00.140496 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7d0cb83-0fb2-48aa-8fb6-2f1af32c97da" containerName="manager" Jul 24 00:30:00.143681 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:00.143659 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747550-69wkt" Jul 24 00:30:00.146469 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:00.146449 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-865ss\"" Jul 24 00:30:00.153831 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:00.153814 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747550-69wkt" Jul 24 00:30:00.159859 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:00.159834 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747550-69wkt"] Jul 24 00:30:00.292491 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:00.292458 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747550-69wkt"] Jul 24 00:30:00.294669 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:30:00.294624 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e4514ed_74e4_4b15_a6c0_d07c42d15a31.slice/crio-e9f3cecab614fc97950aa730b5f6bd93ab151bf86f1d4478c7e401fe2b461fa5 WatchSource:0}: Error finding container e9f3cecab614fc97950aa730b5f6bd93ab151bf86f1d4478c7e401fe2b461fa5: Status 404 returned error can't find the container with id e9f3cecab614fc97950aa730b5f6bd93ab151bf86f1d4478c7e401fe2b461fa5 Jul 24 00:30:00.296533 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:00.296516 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 00:30:00.980583 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:00.980547 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747550-69wkt" event={"ID":"9e4514ed-74e4-4b15-a6c0-d07c42d15a31","Type":"ContainerStarted","Data":"e9f3cecab614fc97950aa730b5f6bd93ab151bf86f1d4478c7e401fe2b461fa5"} Jul 24 00:30:02.992712 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:02.992602 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747550-69wkt" event={"ID":"9e4514ed-74e4-4b15-a6c0-d07c42d15a31","Type":"ContainerStarted","Data":"4f53d4a887eee927dc2b96f5db9332d911b9c609078fe3e93e8fa12461217c1a"} Jul 24 00:30:03.011280 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:03.011225 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747550-69wkt" podStartSLOduration=0.87749113 podStartE2EDuration="3.011206384s" totalImagesPullingTime="2.133715254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:30:03.010131766 +0000 UTC m=+1131.615446086" watchObservedRunningTime="2026-07-24 00:30:03.011206384 +0000 UTC m=+1131.616520747" Jul 24 00:30:03.998896 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:03.998857 2560 generic.go:356] "Generic (PLEG): container finished" podID="9e4514ed-74e4-4b15-a6c0-d07c42d15a31" containerID="4f53d4a887eee927dc2b96f5db9332d911b9c609078fe3e93e8fa12461217c1a" exitCode=0 Jul 24 00:30:03.999272 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:03.998906 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747550-69wkt" event={"ID":"9e4514ed-74e4-4b15-a6c0-d07c42d15a31","Type":"ContainerDied","Data":"4f53d4a887eee927dc2b96f5db9332d911b9c609078fe3e93e8fa12461217c1a"} Jul 24 00:30:05.142411 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:05.142380 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747550-69wkt" Jul 24 00:30:06.009899 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:06.009871 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747550-69wkt" Jul 24 00:30:06.010080 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:06.009910 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747550-69wkt" event={"ID":"9e4514ed-74e4-4b15-a6c0-d07c42d15a31","Type":"ContainerDied","Data":"e9f3cecab614fc97950aa730b5f6bd93ab151bf86f1d4478c7e401fe2b461fa5"} Jul 24 00:30:06.010080 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:06.009953 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9f3cecab614fc97950aa730b5f6bd93ab151bf86f1d4478c7e401fe2b461fa5" Jul 24 00:30:53.812727 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:30:53.812586 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:32:35.574225 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:32:35.574183 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:33:37.918033 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:33:37.917944 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:33:52.967198 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:33:52.967149 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:34:04.793525 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:34:04.793484 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:34:39.511621 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:34:39.511581 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:34:41.237488 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:34:41.237451 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:36:55.042104 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:36:55.042017 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:36:55.129868 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:36:55.129830 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:37:50.136203 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:37:50.136137 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:37:56.163789 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:37:56.163753 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:07.961679 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:07.961565 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:13.809870 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:13.809832 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:17.041020 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:17.040972 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:17.135052 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:17.135020 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:17.592387 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:17.592355 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:23.184597 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.184564 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:23.287830 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.287787 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:23.429093 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.429048 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:23.728497 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.728463 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp"] Jul 24 00:38:23.728878 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.728864 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e4514ed-74e4-4b15-a6c0-d07c42d15a31" containerName="cleanup" Jul 24 00:38:23.728934 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.728879 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e4514ed-74e4-4b15-a6c0-d07c42d15a31" containerName="cleanup" Jul 24 00:38:23.728975 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.728953 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e4514ed-74e4-4b15-a6c0-d07c42d15a31" containerName="cleanup" Jul 24 00:38:23.731970 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.731954 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.734833 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.734812 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-42e5f403-openshift-default-dockercfg-xnpnv\"" Jul 24 00:38:23.751622 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.751588 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp"] Jul 24 00:38:23.754383 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.754352 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-workload-socket\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.754551 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.754389 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-token\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.754551 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.754414 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-workload-certs\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.754551 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.754499 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npclm\" (UniqueName: \"kubernetes.io/projected/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-kube-api-access-npclm\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.754551 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.754536 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-credential-socket\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.754797 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.754589 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-envoy\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.754797 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.754613 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-data\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.754797 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.754702 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istiod-ca-cert\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.754797 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.754743 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-podinfo\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.755514 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.755492 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:23.855533 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.855491 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-data\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.855533 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.855536 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istiod-ca-cert\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.855817 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.855557 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-podinfo\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.855817 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.855606 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-workload-socket\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.855817 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.855626 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-token\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.855817 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.855687 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-workload-certs\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.855817 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.855737 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npclm\" (UniqueName: \"kubernetes.io/projected/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-kube-api-access-npclm\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.855817 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.855765 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-credential-socket\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.855817 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.855815 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-envoy\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.856227 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.855992 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-data\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.856227 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.856059 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-workload-socket\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.856227 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.856168 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-workload-certs\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.856346 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.856232 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-credential-socket\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.856543 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.856519 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istiod-ca-cert\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.858152 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.858131 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-envoy\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.858255 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.858166 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-podinfo\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.864325 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.864294 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-token\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.864430 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.864414 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npclm\" (UniqueName: \"kubernetes.io/projected/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-kube-api-access-npclm\") pod \"e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:23.886094 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:23.886063 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:24.042803 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:24.042765 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:24.184520 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:24.184493 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp"] Jul 24 00:38:24.191041 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:24.191000 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:38:24.191348 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:24.191095 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:38:24.191348 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:24.191129 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:38:24.472179 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:24.472083 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" event={"ID":"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0","Type":"ContainerStarted","Data":"ee6fc763a303fb969dcc0faf8130870c3e96828153db08f0b60ed0ec70c5f95b"} Jul 24 00:38:24.472179 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:24.472138 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" event={"ID":"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0","Type":"ContainerStarted","Data":"e874312dfb0fd32490200fb33e7b74682d95663a4f590021542221fd71f54f25"} Jul 24 00:38:24.496049 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:24.495991 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" podStartSLOduration=1.495972302 podStartE2EDuration="1.495972302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:38:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:38:24.493911203 +0000 UTC m=+1633.099225520" watchObservedRunningTime="2026-07-24 00:38:24.495972302 +0000 UTC m=+1633.101286618" Jul 24 00:38:25.043396 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:25.043348 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:25.048437 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:25.048409 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:25.477129 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:25.477040 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:25.478204 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:25.478174 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:25.904619 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:25.904564 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:26.013007 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:26.012973 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:26.103833 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:26.103778 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:26.256199 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:26.256166 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:26.548684 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:26.548574 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:26.649105 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:26.649068 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:29.819235 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:29.819201 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:29.878809 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:29.878769 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:38:38.544178 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:38.544145 2560 generic.go:356] "Generic (PLEG): container finished" podID="75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" containerID="ee6fc763a303fb969dcc0faf8130870c3e96828153db08f0b60ed0ec70c5f95b" exitCode=0 Jul 24 00:38:38.544614 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:38.544190 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" event={"ID":"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0","Type":"ContainerDied","Data":"ee6fc763a303fb969dcc0faf8130870c3e96828153db08f0b60ed0ec70c5f95b"} Jul 24 00:38:38.544614 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:38.544563 2560 scope.go:117] "RemoveContainer" containerID="ee6fc763a303fb969dcc0faf8130870c3e96828153db08f0b60ed0ec70c5f95b" Jul 24 00:38:38.545592 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:38.545557 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:38:38.545667 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:38.545632 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:38:38.545712 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:38.545680 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:38:39.551345 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:39.551312 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" event={"ID":"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0","Type":"ContainerStarted","Data":"e62076967c3bb0446e847025071f3edb06d1f92867fdd0feff522f1df232d3bd"} Jul 24 00:38:40.043113 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:40.043061 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:40.044216 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:40.044188 2560 patch_prober.go:28] interesting pod/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 24 00:38:40.044329 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:40.044284 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" podUID="75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 24 00:38:40.477482 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:40.477371 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:41.043547 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:41.043504 2560 patch_prober.go:28] interesting pod/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 24 00:38:41.044036 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:41.043579 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" podUID="75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 24 00:38:42.044146 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:42.044109 2560 patch_prober.go:28] interesting pod/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 24 00:38:42.044568 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:42.044189 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" podUID="75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 24 00:38:43.043832 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:43.043793 2560 patch_prober.go:28] interesting pod/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 24 00:38:43.044027 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:43.043851 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" podUID="75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 24 00:38:44.047346 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:44.047310 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:38:44.048306 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:38:44.048290 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:39:02.025560 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:02.025522 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:02.179162 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:02.179125 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:02.263940 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:02.263910 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:06.650653 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:06.650585 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:06.795604 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:06.795571 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp"] Jul 24 00:39:06.795871 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:06.795842 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" podUID="75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" containerName="istio-proxy" containerID="cri-o://e62076967c3bb0446e847025071f3edb06d1f92867fdd0feff522f1df232d3bd" gracePeriod=30 Jul 24 00:39:06.818859 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:06.818820 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:07.080131 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:07.080091 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:07.349936 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:07.349846 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:07.449857 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:07.449819 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:07.569681 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:07.569621 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:07.917442 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:07.917397 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:08.023748 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:08.023715 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:10.799424 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:10.799393 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:10.943679 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:10.943625 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:11.473897 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:11.473867 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:11.574564 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:11.574532 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:11.659965 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:11.659928 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:12.056805 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.056737 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:39:12.142671 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.142601 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-podinfo\") pod \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " Jul 24 00:39:12.142876 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.142689 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-workload-certs\") pod \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " Jul 24 00:39:12.142876 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.142715 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-envoy\") pod \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " Jul 24 00:39:12.142876 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.142735 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-token\") pod \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " Jul 24 00:39:12.142876 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.142765 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istiod-ca-cert\") pod \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " Jul 24 00:39:12.142876 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.142825 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-data\") pod \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " Jul 24 00:39:12.142876 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.142847 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-credential-socket\") pod \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " Jul 24 00:39:12.143210 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.142880 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npclm\" (UniqueName: \"kubernetes.io/projected/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-kube-api-access-npclm\") pod \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " Jul 24 00:39:12.143210 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.142917 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-workload-socket\") pod \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\" (UID: \"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0\") " Jul 24 00:39:12.143210 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.143012 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" (UID: "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:39:12.143210 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.143169 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" (UID: "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:39:12.143423 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.143222 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" (UID: "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:39:12.143423 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.143286 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-credential-socket\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:39:12.143423 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.143308 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-workload-certs\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:39:12.143577 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.143440 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" (UID: "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:39:12.143860 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.143834 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" (UID: "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:39:12.145090 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.145063 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" (UID: "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 00:39:12.145212 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.145159 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" (UID: "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:39:12.145279 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.145232 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" (UID: "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:39:12.145279 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.145246 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-kube-api-access-npclm" (OuterVolumeSpecName: "kube-api-access-npclm") pod "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" (UID: "75dc0fbd-2d67-4c59-a432-5b464eb0b4c0"). InnerVolumeSpecName "kube-api-access-npclm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:39:12.244902 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.244860 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istiod-ca-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:39:12.244902 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.244893 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-data\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:39:12.244902 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.244902 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npclm\" (UniqueName: \"kubernetes.io/projected/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-kube-api-access-npclm\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:39:12.244902 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.244913 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-workload-socket\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:39:12.245181 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.244921 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-podinfo\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:39:12.245181 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.244930 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-envoy\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:39:12.245181 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.244937 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0-istio-token\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:39:12.717968 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.717882 2560 generic.go:356] "Generic (PLEG): container finished" podID="75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" containerID="e62076967c3bb0446e847025071f3edb06d1f92867fdd0feff522f1df232d3bd" exitCode=0 Jul 24 00:39:12.717968 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.717926 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" Jul 24 00:39:12.717968 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.717958 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" event={"ID":"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0","Type":"ContainerDied","Data":"e62076967c3bb0446e847025071f3edb06d1f92867fdd0feff522f1df232d3bd"} Jul 24 00:39:12.718203 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.718011 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp" event={"ID":"75dc0fbd-2d67-4c59-a432-5b464eb0b4c0","Type":"ContainerDied","Data":"e874312dfb0fd32490200fb33e7b74682d95663a4f590021542221fd71f54f25"} Jul 24 00:39:12.718203 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.718033 2560 scope.go:117] "RemoveContainer" containerID="e62076967c3bb0446e847025071f3edb06d1f92867fdd0feff522f1df232d3bd" Jul 24 00:39:12.727035 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.727017 2560 scope.go:117] "RemoveContainer" containerID="ee6fc763a303fb969dcc0faf8130870c3e96828153db08f0b60ed0ec70c5f95b" Jul 24 00:39:12.735586 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.735549 2560 scope.go:117] "RemoveContainer" containerID="e62076967c3bb0446e847025071f3edb06d1f92867fdd0feff522f1df232d3bd" Jul 24 00:39:12.735931 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:39:12.735902 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e62076967c3bb0446e847025071f3edb06d1f92867fdd0feff522f1df232d3bd\": container with ID starting with e62076967c3bb0446e847025071f3edb06d1f92867fdd0feff522f1df232d3bd not found: ID does not exist" containerID="e62076967c3bb0446e847025071f3edb06d1f92867fdd0feff522f1df232d3bd" Jul 24 00:39:12.735997 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.735945 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e62076967c3bb0446e847025071f3edb06d1f92867fdd0feff522f1df232d3bd"} err="failed to get container status \"e62076967c3bb0446e847025071f3edb06d1f92867fdd0feff522f1df232d3bd\": rpc error: code = NotFound desc = could not find container \"e62076967c3bb0446e847025071f3edb06d1f92867fdd0feff522f1df232d3bd\": container with ID starting with e62076967c3bb0446e847025071f3edb06d1f92867fdd0feff522f1df232d3bd not found: ID does not exist" Jul 24 00:39:12.735997 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.735968 2560 scope.go:117] "RemoveContainer" containerID="ee6fc763a303fb969dcc0faf8130870c3e96828153db08f0b60ed0ec70c5f95b" Jul 24 00:39:12.736229 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:39:12.736212 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee6fc763a303fb969dcc0faf8130870c3e96828153db08f0b60ed0ec70c5f95b\": container with ID starting with ee6fc763a303fb969dcc0faf8130870c3e96828153db08f0b60ed0ec70c5f95b not found: ID does not exist" containerID="ee6fc763a303fb969dcc0faf8130870c3e96828153db08f0b60ed0ec70c5f95b" Jul 24 00:39:12.736288 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.736240 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee6fc763a303fb969dcc0faf8130870c3e96828153db08f0b60ed0ec70c5f95b"} err="failed to get container status \"ee6fc763a303fb969dcc0faf8130870c3e96828153db08f0b60ed0ec70c5f95b\": rpc error: code = NotFound desc = could not find container \"ee6fc763a303fb969dcc0faf8130870c3e96828153db08f0b60ed0ec70c5f95b\": container with ID starting with ee6fc763a303fb969dcc0faf8130870c3e96828153db08f0b60ed0ec70c5f95b not found: ID does not exist" Jul 24 00:39:12.736288 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.736271 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp"] Jul 24 00:39:12.742612 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:12.742584 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-42e5f403-openshift-default-6f95f5f96b-zr4cp"] Jul 24 00:39:14.348432 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:14.348394 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" path="/var/lib/kubelet/pods/75dc0fbd-2d67-4c59-a432-5b464eb0b4c0/volumes" Jul 24 00:39:35.881143 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:35.881108 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5"] Jul 24 00:39:35.881618 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:35.881521 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" containerName="istio-proxy" Jul 24 00:39:35.881618 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:35.881532 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" containerName="istio-proxy" Jul 24 00:39:35.881618 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:35.881548 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" containerName="istio-proxy" Jul 24 00:39:35.881618 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:35.881554 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" containerName="istio-proxy" Jul 24 00:39:35.881618 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:35.881609 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" containerName="istio-proxy" Jul 24 00:39:35.881618 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:35.881618 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="75dc0fbd-2d67-4c59-a432-5b464eb0b4c0" containerName="istio-proxy" Jul 24 00:39:35.886102 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:35.886082 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5" Jul 24 00:39:35.888735 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:35.888710 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-865ss\"" Jul 24 00:39:35.888873 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:35.888792 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 00:39:35.893438 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:35.893407 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5"] Jul 24 00:39:35.986881 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:35.986843 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d96202df-ebe1-493b-9805-55c680dadf15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5\" (UID: \"d96202df-ebe1-493b-9805-55c680dadf15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5" Jul 24 00:39:36.087954 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:36.087917 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d96202df-ebe1-493b-9805-55c680dadf15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5\" (UID: \"d96202df-ebe1-493b-9805-55c680dadf15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5" Jul 24 00:39:36.088543 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:36.088521 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d96202df-ebe1-493b-9805-55c680dadf15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5\" (UID: \"d96202df-ebe1-493b-9805-55c680dadf15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5" Jul 24 00:39:36.198095 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:36.198011 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5" Jul 24 00:39:36.337162 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:36.337132 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5"] Jul 24 00:39:36.339366 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:39:36.339337 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd96202df_ebe1_493b_9805_55c680dadf15.slice/crio-6229716dd4d04a1ed63f2b1b0c4ee461df64c95cb1a896f37f37fef09d5e0345 WatchSource:0}: Error finding container 6229716dd4d04a1ed63f2b1b0c4ee461df64c95cb1a896f37f37fef09d5e0345: Status 404 returned error can't find the container with id 6229716dd4d04a1ed63f2b1b0c4ee461df64c95cb1a896f37f37fef09d5e0345 Jul 24 00:39:36.838312 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:36.838221 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5" event={"ID":"d96202df-ebe1-493b-9805-55c680dadf15","Type":"ContainerStarted","Data":"26c534f2d321312310f4862e2566383dc0e9e123dbd360fd31f55434fb6b7a4d"} Jul 24 00:39:36.838312 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:36.838282 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5" event={"ID":"d96202df-ebe1-493b-9805-55c680dadf15","Type":"ContainerStarted","Data":"6229716dd4d04a1ed63f2b1b0c4ee461df64c95cb1a896f37f37fef09d5e0345"} Jul 24 00:39:36.855506 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:36.855451 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5" podStartSLOduration=1.85543364 podStartE2EDuration="1.85543364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:39:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:39:36.854129569 +0000 UTC m=+1705.459443876" watchObservedRunningTime="2026-07-24 00:39:36.85543364 +0000 UTC m=+1705.460747994" Jul 24 00:39:37.845209 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:37.845174 2560 generic.go:356] "Generic (PLEG): container finished" podID="d96202df-ebe1-493b-9805-55c680dadf15" containerID="26c534f2d321312310f4862e2566383dc0e9e123dbd360fd31f55434fb6b7a4d" exitCode=0 Jul 24 00:39:37.845607 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:37.845223 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5" event={"ID":"d96202df-ebe1-493b-9805-55c680dadf15","Type":"ContainerDied","Data":"26c534f2d321312310f4862e2566383dc0e9e123dbd360fd31f55434fb6b7a4d"} Jul 24 00:39:38.987599 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:38.987575 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5" Jul 24 00:39:39.018120 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:39.018085 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d96202df-ebe1-493b-9805-55c680dadf15-maas-api-service-ca\") pod \"d96202df-ebe1-493b-9805-55c680dadf15\" (UID: \"d96202df-ebe1-493b-9805-55c680dadf15\") " Jul 24 00:39:39.018444 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:39.018423 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d96202df-ebe1-493b-9805-55c680dadf15-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d96202df-ebe1-493b-9805-55c680dadf15" (UID: "d96202df-ebe1-493b-9805-55c680dadf15"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:39:39.119930 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:39.119843 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d96202df-ebe1-493b-9805-55c680dadf15-maas-api-service-ca\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:39:39.857344 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:39.857314 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5" Jul 24 00:39:39.857344 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:39.857336 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5" event={"ID":"d96202df-ebe1-493b-9805-55c680dadf15","Type":"ContainerDied","Data":"6229716dd4d04a1ed63f2b1b0c4ee461df64c95cb1a896f37f37fef09d5e0345"} Jul 24 00:39:39.857559 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:39.857378 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6229716dd4d04a1ed63f2b1b0c4ee461df64c95cb1a896f37f37fef09d5e0345" Jul 24 00:39:46.390289 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:46.390256 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:46.490151 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:46.490117 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:46.589537 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:46.589508 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:50.955211 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:50.955136 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5"] Jul 24 00:39:50.957842 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:50.957817 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8405249f-2d6m5"] Jul 24 00:39:50.957977 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:50.957961 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:51.113896 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:51.113808 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:51.364205 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:51.364173 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:51.621949 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:51.621869 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:51.926784 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:51.926618 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7"] Jul 24 00:39:51.927282 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:51.927261 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d96202df-ebe1-493b-9805-55c680dadf15" containerName="revoke-keys" Jul 24 00:39:51.927282 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:51.927283 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d96202df-ebe1-493b-9805-55c680dadf15" containerName="revoke-keys" Jul 24 00:39:51.927446 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:51.927372 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d96202df-ebe1-493b-9805-55c680dadf15" containerName="revoke-keys" Jul 24 00:39:51.930360 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:51.930340 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:51.933137 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:51.933113 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-dd068a7a-openshift-default-dockercfg-4xjp2\"" Jul 24 00:39:51.943748 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:51.943718 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7"] Jul 24 00:39:51.964349 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:51.964318 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:52.047787 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.047754 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-workload-certs\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.047967 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.047795 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-workload-socket\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.047967 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.047842 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/605a700c-1246-4937-9006-7ab2cad03bd5-istio-podinfo\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.047967 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.047891 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/605a700c-1246-4937-9006-7ab2cad03bd5-istiod-ca-cert\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.047967 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.047928 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-istio-data\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.047967 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.047948 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7296r\" (UniqueName: \"kubernetes.io/projected/605a700c-1246-4937-9006-7ab2cad03bd5-kube-api-access-7296r\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.048157 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.048025 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-credential-socket\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.048157 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.048101 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-istio-envoy\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.048157 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.048131 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/605a700c-1246-4937-9006-7ab2cad03bd5-istio-token\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.063384 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.063338 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:52.148757 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.148723 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-istio-envoy\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.148757 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.148767 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/605a700c-1246-4937-9006-7ab2cad03bd5-istio-token\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.149003 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.148811 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-workload-certs\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.149003 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.148833 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-workload-socket\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.149003 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.148870 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/605a700c-1246-4937-9006-7ab2cad03bd5-istio-podinfo\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.149003 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.148896 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/605a700c-1246-4937-9006-7ab2cad03bd5-istiod-ca-cert\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.149003 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.148929 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-istio-data\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.149003 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.148955 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7296r\" (UniqueName: \"kubernetes.io/projected/605a700c-1246-4937-9006-7ab2cad03bd5-kube-api-access-7296r\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.149003 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.149002 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-credential-socket\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.149357 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.149275 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-workload-certs\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.149357 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.149287 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-workload-socket\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.149606 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.149566 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-istio-data\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.149734 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.149715 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-credential-socket\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.149778 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.149752 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/605a700c-1246-4937-9006-7ab2cad03bd5-istiod-ca-cert\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.151036 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.151011 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-istio-envoy\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.151401 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.151381 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/605a700c-1246-4937-9006-7ab2cad03bd5-istio-podinfo\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.157344 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.157313 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/605a700c-1246-4937-9006-7ab2cad03bd5-istio-token\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.157457 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.157396 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7296r\" (UniqueName: \"kubernetes.io/projected/605a700c-1246-4937-9006-7ab2cad03bd5-kube-api-access-7296r\") pod \"e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.241800 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.241701 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:52.357618 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.357577 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d96202df-ebe1-493b-9805-55c680dadf15" path="/var/lib/kubelet/pods/d96202df-ebe1-493b-9805-55c680dadf15/volumes" Jul 24 00:39:52.388972 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.388947 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7"] Jul 24 00:39:52.390804 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:39:52.390773 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod605a700c_1246_4937_9006_7ab2cad03bd5.slice/crio-6e29091555b10675298abd44c1ababe086d9a09414955749cf2f0c33382c566f WatchSource:0}: Error finding container 6e29091555b10675298abd44c1ababe086d9a09414955749cf2f0c33382c566f: Status 404 returned error can't find the container with id 6e29091555b10675298abd44c1ababe086d9a09414955749cf2f0c33382c566f Jul 24 00:39:52.393123 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.393063 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:39:52.393224 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.393173 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:39:52.393279 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.393221 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:39:52.924824 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.924787 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" event={"ID":"605a700c-1246-4937-9006-7ab2cad03bd5","Type":"ContainerStarted","Data":"70689ccbc17b81a0231620a1667545a6e0ea76059a9a6eaf174659aead69148c"} Jul 24 00:39:52.924991 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.924847 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" event={"ID":"605a700c-1246-4937-9006-7ab2cad03bd5","Type":"ContainerStarted","Data":"6e29091555b10675298abd44c1ababe086d9a09414955749cf2f0c33382c566f"} Jul 24 00:39:52.946401 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:52.946349 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" podStartSLOduration=1.946330792 podStartE2EDuration="1.946330792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:39:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:39:52.945409245 +0000 UTC m=+1721.550723574" watchObservedRunningTime="2026-07-24 00:39:52.946330792 +0000 UTC m=+1721.551645107" Jul 24 00:39:53.242261 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:53.242169 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:53.246978 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:53.246952 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:53.929986 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:53.929944 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:53.931069 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:53.931049 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:39:54.289259 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:54.289225 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:54.389716 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:54.389686 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:54.504876 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:54.504845 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:54.710130 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:54.710040 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:54.966459 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:54.966376 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:39:55.069427 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:39:55.069392 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:09.971866 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:09.971807 2560 patch_prober.go:28] interesting pod/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 00:40:09.972466 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:09.971886 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" podUID="605a700c-1246-4937-9006-7ab2cad03bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 00:40:11.024260 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:11.024221 2560 generic.go:356] "Generic (PLEG): container finished" podID="605a700c-1246-4937-9006-7ab2cad03bd5" containerID="70689ccbc17b81a0231620a1667545a6e0ea76059a9a6eaf174659aead69148c" exitCode=0 Jul 24 00:40:11.024830 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:11.024299 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" event={"ID":"605a700c-1246-4937-9006-7ab2cad03bd5","Type":"ContainerDied","Data":"70689ccbc17b81a0231620a1667545a6e0ea76059a9a6eaf174659aead69148c"} Jul 24 00:40:11.024948 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:11.024929 2560 scope.go:117] "RemoveContainer" containerID="70689ccbc17b81a0231620a1667545a6e0ea76059a9a6eaf174659aead69148c" Jul 24 00:40:11.025963 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:11.025937 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:40:11.026025 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:11.026011 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:40:11.026068 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:11.026055 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:40:12.031337 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:12.031297 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" event={"ID":"605a700c-1246-4937-9006-7ab2cad03bd5","Type":"ContainerStarted","Data":"d0d28fc8c00ec0f0553741bb52569db81abbe7a2b59f66376fb5a85de45590d3"} Jul 24 00:40:12.242196 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:12.242161 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:40:12.242413 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:12.242395 2560 patch_prober.go:28] interesting pod/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 24 00:40:12.242463 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:12.242450 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" podUID="605a700c-1246-4937-9006-7ab2cad03bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 24 00:40:13.243160 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:13.243118 2560 patch_prober.go:28] interesting pod/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 24 00:40:13.243569 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:13.243187 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" podUID="605a700c-1246-4937-9006-7ab2cad03bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 24 00:40:14.242277 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:14.242243 2560 patch_prober.go:28] interesting pod/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 24 00:40:14.242459 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:14.242306 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" podUID="605a700c-1246-4937-9006-7ab2cad03bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 24 00:40:15.242750 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:15.242710 2560 patch_prober.go:28] interesting pod/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 24 00:40:15.243179 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:15.242778 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" podUID="605a700c-1246-4937-9006-7ab2cad03bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 24 00:40:16.242772 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:16.242730 2560 patch_prober.go:28] interesting pod/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 24 00:40:16.243209 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:16.242790 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" podUID="605a700c-1246-4937-9006-7ab2cad03bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 24 00:40:17.246038 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:17.246006 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:40:17.246482 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:17.246280 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:40:17.247075 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:17.247052 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:40:20.231750 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:20.231715 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49"] Jul 24 00:40:20.234170 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:20.234152 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49" Jul 24 00:40:20.236830 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:20.236805 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 00:40:20.237008 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:20.236926 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-865ss\"" Jul 24 00:40:20.242900 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:20.242869 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49"] Jul 24 00:40:20.344694 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:20.344611 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ec8bef-18af-415e-824b-b9a19b203254-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dd068a7a-dws49\" (UID: \"f1ec8bef-18af-415e-824b-b9a19b203254\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49" Jul 24 00:40:20.446241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:20.446202 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ec8bef-18af-415e-824b-b9a19b203254-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dd068a7a-dws49\" (UID: \"f1ec8bef-18af-415e-824b-b9a19b203254\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49" Jul 24 00:40:20.446959 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:20.446934 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ec8bef-18af-415e-824b-b9a19b203254-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dd068a7a-dws49\" (UID: \"f1ec8bef-18af-415e-824b-b9a19b203254\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49" Jul 24 00:40:20.546357 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:20.546321 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49" Jul 24 00:40:20.683790 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:20.683754 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49"] Jul 24 00:40:20.691315 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:40:20.691284 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ec8bef_18af_415e_824b_b9a19b203254.slice/crio-edcfb4346606572f5bcf137a02a8d78d7779f0962dea172624c81d4103a771a4 WatchSource:0}: Error finding container edcfb4346606572f5bcf137a02a8d78d7779f0962dea172624c81d4103a771a4: Status 404 returned error can't find the container with id edcfb4346606572f5bcf137a02a8d78d7779f0962dea172624c81d4103a771a4 Jul 24 00:40:21.079162 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:21.079122 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49" event={"ID":"f1ec8bef-18af-415e-824b-b9a19b203254","Type":"ContainerStarted","Data":"3c3dfd95e0173b86d65a16f5b462022523a61522bf56881d1a36e66bb37290cc"} Jul 24 00:40:21.079346 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:21.079183 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49" event={"ID":"f1ec8bef-18af-415e-824b-b9a19b203254","Type":"ContainerStarted","Data":"edcfb4346606572f5bcf137a02a8d78d7779f0962dea172624c81d4103a771a4"} Jul 24 00:40:21.097368 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:21.097312 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49" podStartSLOduration=1.097297981 podStartE2EDuration="1.097297981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:40:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:40:21.096593256 +0000 UTC m=+1749.701907573" watchObservedRunningTime="2026-07-24 00:40:21.097297981 +0000 UTC m=+1749.702612297" Jul 24 00:40:22.085030 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:22.084997 2560 generic.go:356] "Generic (PLEG): container finished" podID="f1ec8bef-18af-415e-824b-b9a19b203254" containerID="3c3dfd95e0173b86d65a16f5b462022523a61522bf56881d1a36e66bb37290cc" exitCode=0 Jul 24 00:40:22.085419 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:22.085080 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49" event={"ID":"f1ec8bef-18af-415e-824b-b9a19b203254","Type":"ContainerDied","Data":"3c3dfd95e0173b86d65a16f5b462022523a61522bf56881d1a36e66bb37290cc"} Jul 24 00:40:23.231741 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:23.231711 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49" Jul 24 00:40:23.375591 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:23.375494 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ec8bef-18af-415e-824b-b9a19b203254-maas-api-service-ca\") pod \"f1ec8bef-18af-415e-824b-b9a19b203254\" (UID: \"f1ec8bef-18af-415e-824b-b9a19b203254\") " Jul 24 00:40:23.375904 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:23.375878 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1ec8bef-18af-415e-824b-b9a19b203254-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f1ec8bef-18af-415e-824b-b9a19b203254" (UID: "f1ec8bef-18af-415e-824b-b9a19b203254"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:40:23.476666 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:23.476603 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1ec8bef-18af-415e-824b-b9a19b203254-maas-api-service-ca\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:40:24.095973 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:24.095942 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49" Jul 24 00:40:24.096166 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:24.095986 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49" event={"ID":"f1ec8bef-18af-415e-824b-b9a19b203254","Type":"ContainerDied","Data":"edcfb4346606572f5bcf137a02a8d78d7779f0962dea172624c81d4103a771a4"} Jul 24 00:40:24.096166 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:24.096041 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="edcfb4346606572f5bcf137a02a8d78d7779f0962dea172624c81d4103a771a4" Jul 24 00:40:30.762965 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:30.762933 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:30.838837 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:30.838803 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:30.912842 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:30.912805 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:35.309994 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:35.309955 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:35.326364 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:35.326326 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49"] Jul 24 00:40:35.330341 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:35.330305 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dd068a7a-dws49"] Jul 24 00:40:36.348471 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:36.348434 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1ec8bef-18af-415e-824b-b9a19b203254" path="/var/lib/kubelet/pods/f1ec8bef-18af-415e-824b-b9a19b203254/volumes" Jul 24 00:40:36.543742 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:36.543709 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7"] Jul 24 00:40:36.544065 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:36.544014 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" podUID="605a700c-1246-4937-9006-7ab2cad03bd5" containerName="istio-proxy" containerID="cri-o://d0d28fc8c00ec0f0553741bb52569db81abbe7a2b59f66376fb5a85de45590d3" gracePeriod=30 Jul 24 00:40:36.565069 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:36.565030 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:36.803082 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:36.803050 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:37.050257 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:37.050222 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:37.388927 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:37.388895 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:37.805211 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:37.805173 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:40.090723 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:40.090682 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:40.177799 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:40.177767 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:40.255923 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:40.255890 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:40.505470 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:40.505438 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:40.778897 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:40.778810 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:40.879083 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:40.879052 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:40:41.804008 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.803932 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:40:41.867811 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.867774 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-istio-envoy\") pod \"605a700c-1246-4937-9006-7ab2cad03bd5\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " Jul 24 00:40:41.867811 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.867816 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/605a700c-1246-4937-9006-7ab2cad03bd5-istiod-ca-cert\") pod \"605a700c-1246-4937-9006-7ab2cad03bd5\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " Jul 24 00:40:41.868064 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.867859 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7296r\" (UniqueName: \"kubernetes.io/projected/605a700c-1246-4937-9006-7ab2cad03bd5-kube-api-access-7296r\") pod \"605a700c-1246-4937-9006-7ab2cad03bd5\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " Jul 24 00:40:41.868064 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.867904 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-istio-data\") pod \"605a700c-1246-4937-9006-7ab2cad03bd5\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " Jul 24 00:40:41.868064 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.867940 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-workload-certs\") pod \"605a700c-1246-4937-9006-7ab2cad03bd5\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " Jul 24 00:40:41.868064 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.868020 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/605a700c-1246-4937-9006-7ab2cad03bd5-istio-podinfo\") pod \"605a700c-1246-4937-9006-7ab2cad03bd5\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " Jul 24 00:40:41.868263 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.868135 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-credential-socket\") pod \"605a700c-1246-4937-9006-7ab2cad03bd5\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " Jul 24 00:40:41.868263 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.868162 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/605a700c-1246-4937-9006-7ab2cad03bd5-istio-token\") pod \"605a700c-1246-4937-9006-7ab2cad03bd5\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " Jul 24 00:40:41.868263 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.868204 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-workload-socket\") pod \"605a700c-1246-4937-9006-7ab2cad03bd5\" (UID: \"605a700c-1246-4937-9006-7ab2cad03bd5\") " Jul 24 00:40:41.868418 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.868291 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "605a700c-1246-4937-9006-7ab2cad03bd5" (UID: "605a700c-1246-4937-9006-7ab2cad03bd5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:40:41.868474 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.868416 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/605a700c-1246-4937-9006-7ab2cad03bd5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "605a700c-1246-4937-9006-7ab2cad03bd5" (UID: "605a700c-1246-4937-9006-7ab2cad03bd5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:40:41.868854 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.868524 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "605a700c-1246-4937-9006-7ab2cad03bd5" (UID: "605a700c-1246-4937-9006-7ab2cad03bd5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:40:41.868854 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.868705 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-workload-socket\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:40:41.868854 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.868712 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "605a700c-1246-4937-9006-7ab2cad03bd5" (UID: "605a700c-1246-4937-9006-7ab2cad03bd5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:40:41.868854 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.868726 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/605a700c-1246-4937-9006-7ab2cad03bd5-istiod-ca-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:40:41.868854 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.868741 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-workload-certs\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:40:41.869113 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.868960 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-istio-data" (OuterVolumeSpecName: "istio-data") pod "605a700c-1246-4937-9006-7ab2cad03bd5" (UID: "605a700c-1246-4937-9006-7ab2cad03bd5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:40:41.870292 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.870260 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/605a700c-1246-4937-9006-7ab2cad03bd5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "605a700c-1246-4937-9006-7ab2cad03bd5" (UID: "605a700c-1246-4937-9006-7ab2cad03bd5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 00:40:41.870593 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.870566 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/605a700c-1246-4937-9006-7ab2cad03bd5-istio-token" (OuterVolumeSpecName: "istio-token") pod "605a700c-1246-4937-9006-7ab2cad03bd5" (UID: "605a700c-1246-4937-9006-7ab2cad03bd5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:40:41.870593 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.870577 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "605a700c-1246-4937-9006-7ab2cad03bd5" (UID: "605a700c-1246-4937-9006-7ab2cad03bd5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:40:41.870593 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.870588 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/605a700c-1246-4937-9006-7ab2cad03bd5-kube-api-access-7296r" (OuterVolumeSpecName: "kube-api-access-7296r") pod "605a700c-1246-4937-9006-7ab2cad03bd5" (UID: "605a700c-1246-4937-9006-7ab2cad03bd5"). InnerVolumeSpecName "kube-api-access-7296r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:40:41.970115 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.970072 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-istio-envoy\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:40:41.970115 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.970106 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7296r\" (UniqueName: \"kubernetes.io/projected/605a700c-1246-4937-9006-7ab2cad03bd5-kube-api-access-7296r\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:40:41.970115 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.970116 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-istio-data\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:40:41.970115 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.970124 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/605a700c-1246-4937-9006-7ab2cad03bd5-istio-podinfo\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:40:41.970402 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.970132 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/605a700c-1246-4937-9006-7ab2cad03bd5-credential-socket\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:40:41.970402 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:41.970140 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/605a700c-1246-4937-9006-7ab2cad03bd5-istio-token\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:40:42.196815 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:42.196730 2560 generic.go:356] "Generic (PLEG): container finished" podID="605a700c-1246-4937-9006-7ab2cad03bd5" containerID="d0d28fc8c00ec0f0553741bb52569db81abbe7a2b59f66376fb5a85de45590d3" exitCode=0 Jul 24 00:40:42.196815 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:42.196779 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" Jul 24 00:40:42.196998 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:42.196807 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" event={"ID":"605a700c-1246-4937-9006-7ab2cad03bd5","Type":"ContainerDied","Data":"d0d28fc8c00ec0f0553741bb52569db81abbe7a2b59f66376fb5a85de45590d3"} Jul 24 00:40:42.196998 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:42.196896 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7" event={"ID":"605a700c-1246-4937-9006-7ab2cad03bd5","Type":"ContainerDied","Data":"6e29091555b10675298abd44c1ababe086d9a09414955749cf2f0c33382c566f"} Jul 24 00:40:42.196998 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:42.196921 2560 scope.go:117] "RemoveContainer" containerID="d0d28fc8c00ec0f0553741bb52569db81abbe7a2b59f66376fb5a85de45590d3" Jul 24 00:40:42.206831 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:42.206805 2560 scope.go:117] "RemoveContainer" containerID="70689ccbc17b81a0231620a1667545a6e0ea76059a9a6eaf174659aead69148c" Jul 24 00:40:42.214982 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:42.214957 2560 scope.go:117] "RemoveContainer" containerID="d0d28fc8c00ec0f0553741bb52569db81abbe7a2b59f66376fb5a85de45590d3" Jul 24 00:40:42.215288 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:40:42.215268 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0d28fc8c00ec0f0553741bb52569db81abbe7a2b59f66376fb5a85de45590d3\": container with ID starting with d0d28fc8c00ec0f0553741bb52569db81abbe7a2b59f66376fb5a85de45590d3 not found: ID does not exist" containerID="d0d28fc8c00ec0f0553741bb52569db81abbe7a2b59f66376fb5a85de45590d3" Jul 24 00:40:42.215337 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:42.215299 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0d28fc8c00ec0f0553741bb52569db81abbe7a2b59f66376fb5a85de45590d3"} err="failed to get container status \"d0d28fc8c00ec0f0553741bb52569db81abbe7a2b59f66376fb5a85de45590d3\": rpc error: code = NotFound desc = could not find container \"d0d28fc8c00ec0f0553741bb52569db81abbe7a2b59f66376fb5a85de45590d3\": container with ID starting with d0d28fc8c00ec0f0553741bb52569db81abbe7a2b59f66376fb5a85de45590d3 not found: ID does not exist" Jul 24 00:40:42.215337 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:42.215318 2560 scope.go:117] "RemoveContainer" containerID="70689ccbc17b81a0231620a1667545a6e0ea76059a9a6eaf174659aead69148c" Jul 24 00:40:42.215568 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:40:42.215550 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70689ccbc17b81a0231620a1667545a6e0ea76059a9a6eaf174659aead69148c\": container with ID starting with 70689ccbc17b81a0231620a1667545a6e0ea76059a9a6eaf174659aead69148c not found: ID does not exist" containerID="70689ccbc17b81a0231620a1667545a6e0ea76059a9a6eaf174659aead69148c" Jul 24 00:40:42.215616 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:42.215574 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70689ccbc17b81a0231620a1667545a6e0ea76059a9a6eaf174659aead69148c"} err="failed to get container status \"70689ccbc17b81a0231620a1667545a6e0ea76059a9a6eaf174659aead69148c\": rpc error: code = NotFound desc = could not find container \"70689ccbc17b81a0231620a1667545a6e0ea76059a9a6eaf174659aead69148c\": container with ID starting with 70689ccbc17b81a0231620a1667545a6e0ea76059a9a6eaf174659aead69148c not found: ID does not exist" Jul 24 00:40:42.218709 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:42.218678 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7"] Jul 24 00:40:42.222602 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:42.222574 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-dd068a7a-openshift-default-5db6875b54-np7x7"] Jul 24 00:40:42.348201 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:40:42.348168 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="605a700c-1246-4937-9006-7ab2cad03bd5" path="/var/lib/kubelet/pods/605a700c-1246-4937-9006-7ab2cad03bd5/volumes" Jul 24 00:41:05.038125 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.038037 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw"] Jul 24 00:41:05.038531 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.038427 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="605a700c-1246-4937-9006-7ab2cad03bd5" containerName="istio-proxy" Jul 24 00:41:05.038531 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.038439 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="605a700c-1246-4937-9006-7ab2cad03bd5" containerName="istio-proxy" Jul 24 00:41:05.038531 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.038451 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="605a700c-1246-4937-9006-7ab2cad03bd5" containerName="istio-proxy" Jul 24 00:41:05.038531 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.038457 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="605a700c-1246-4937-9006-7ab2cad03bd5" containerName="istio-proxy" Jul 24 00:41:05.038531 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.038466 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1ec8bef-18af-415e-824b-b9a19b203254" containerName="revoke-keys" Jul 24 00:41:05.038531 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.038472 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1ec8bef-18af-415e-824b-b9a19b203254" containerName="revoke-keys" Jul 24 00:41:05.038820 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.038544 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="605a700c-1246-4937-9006-7ab2cad03bd5" containerName="istio-proxy" Jul 24 00:41:05.038820 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.038550 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="605a700c-1246-4937-9006-7ab2cad03bd5" containerName="istio-proxy" Jul 24 00:41:05.038820 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.038558 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1ec8bef-18af-415e-824b-b9a19b203254" containerName="revoke-keys" Jul 24 00:41:05.040938 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.040921 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw" Jul 24 00:41:05.044002 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.043976 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-865ss\"" Jul 24 00:41:05.044863 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.044848 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 00:41:05.049778 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.049755 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw"] Jul 24 00:41:05.191522 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.191467 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9efe9f4c-49de-489f-8fb8-5182ccfa0607-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw\" (UID: \"9efe9f4c-49de-489f-8fb8-5182ccfa0607\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw" Jul 24 00:41:05.292514 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.292419 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9efe9f4c-49de-489f-8fb8-5182ccfa0607-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw\" (UID: \"9efe9f4c-49de-489f-8fb8-5182ccfa0607\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw" Jul 24 00:41:05.293063 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.293041 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9efe9f4c-49de-489f-8fb8-5182ccfa0607-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw\" (UID: \"9efe9f4c-49de-489f-8fb8-5182ccfa0607\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw" Jul 24 00:41:05.353272 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.353235 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw" Jul 24 00:41:05.487944 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:05.487919 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw"] Jul 24 00:41:05.489763 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:41:05.489734 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efe9f4c_49de_489f_8fb8_5182ccfa0607.slice/crio-0ac362a01a5e61019304cdc293ffd2e0b321dde469e29a7e71ab018fb4157a54 WatchSource:0}: Error finding container 0ac362a01a5e61019304cdc293ffd2e0b321dde469e29a7e71ab018fb4157a54: Status 404 returned error can't find the container with id 0ac362a01a5e61019304cdc293ffd2e0b321dde469e29a7e71ab018fb4157a54 Jul 24 00:41:06.328520 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:06.328477 2560 generic.go:356] "Generic (PLEG): container finished" podID="9efe9f4c-49de-489f-8fb8-5182ccfa0607" containerID="c42887026df2e9098c96a512c7da2cbff1e2dd5fe81a0f071ba06d7d3d31f6d0" exitCode=0 Jul 24 00:41:06.328989 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:06.328571 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw" event={"ID":"9efe9f4c-49de-489f-8fb8-5182ccfa0607","Type":"ContainerDied","Data":"c42887026df2e9098c96a512c7da2cbff1e2dd5fe81a0f071ba06d7d3d31f6d0"} Jul 24 00:41:06.328989 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:06.328628 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw" event={"ID":"9efe9f4c-49de-489f-8fb8-5182ccfa0607","Type":"ContainerStarted","Data":"0ac362a01a5e61019304cdc293ffd2e0b321dde469e29a7e71ab018fb4157a54"} Jul 24 00:41:07.467449 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:07.467422 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw" Jul 24 00:41:07.612404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:07.612314 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9efe9f4c-49de-489f-8fb8-5182ccfa0607-maas-api-service-ca\") pod \"9efe9f4c-49de-489f-8fb8-5182ccfa0607\" (UID: \"9efe9f4c-49de-489f-8fb8-5182ccfa0607\") " Jul 24 00:41:07.612658 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:07.612613 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9efe9f4c-49de-489f-8fb8-5182ccfa0607-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9efe9f4c-49de-489f-8fb8-5182ccfa0607" (UID: "9efe9f4c-49de-489f-8fb8-5182ccfa0607"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:41:07.713822 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:07.713781 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9efe9f4c-49de-489f-8fb8-5182ccfa0607-maas-api-service-ca\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:41:08.339849 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:08.339807 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw" event={"ID":"9efe9f4c-49de-489f-8fb8-5182ccfa0607","Type":"ContainerDied","Data":"0ac362a01a5e61019304cdc293ffd2e0b321dde469e29a7e71ab018fb4157a54"} Jul 24 00:41:08.340026 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:08.339884 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ac362a01a5e61019304cdc293ffd2e0b321dde469e29a7e71ab018fb4157a54" Jul 24 00:41:08.340026 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:08.339816 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw" Jul 24 00:41:15.516535 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:15.516495 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:41:15.615997 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:15.615960 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:41:20.117673 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:20.117596 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:41:20.118195 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:20.117873 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw"] Jul 24 00:41:20.120126 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:20.120098 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bd6ad2bc-b2qdw"] Jul 24 00:41:20.279626 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:20.279590 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:41:20.347817 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:20.347784 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9efe9f4c-49de-489f-8fb8-5182ccfa0607" path="/var/lib/kubelet/pods/9efe9f4c-49de-489f-8fb8-5182ccfa0607/volumes" Jul 24 00:41:20.520387 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:20.520355 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:41:21.053256 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:21.053219 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:41:21.149263 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:21.149228 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:41:21.249812 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:21.249772 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:41:22.764156 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:22.764111 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:41:28.490443 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:28.490406 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:41:28.550529 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:28.550500 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:41:28.623967 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:28.623932 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:41:28.725304 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:28.725268 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:41:28.784691 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:28.784633 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:41:29.251693 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:29.251664 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:41:55.258166 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:41:55.258125 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:01.578747 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:01.578702 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:01.693342 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:01.693304 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:01.782973 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:01.782935 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:17.699661 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:17.699596 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:23.477159 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:23.477121 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:23.588498 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:23.588457 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:23.681192 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:23.681154 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:34.218592 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:34.218483 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:35.034395 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:35.034361 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:41.718962 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:41.718924 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:41.821154 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:41.821117 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:41.919272 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:41.919240 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:42.019611 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:42.019581 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:42.827131 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:42.827097 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:42.934101 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:42.934062 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:43.030453 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:43.030417 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:44.727762 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:44.727728 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:50.708564 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:50.708523 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:50.809576 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:50.809540 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:51.406420 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:51.406385 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:57.210821 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:57.210771 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:42:58.405958 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:42:58.405923 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:00.670997 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:00.670960 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:00.769808 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:00.769775 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:00.893057 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:00.893023 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:01.127925 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.127888 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:01.486277 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.486182 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:01.588266 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.588236 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:01.625829 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.625779 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr"] Jul 24 00:43:01.626411 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.626392 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9efe9f4c-49de-489f-8fb8-5182ccfa0607" containerName="revoke-keys" Jul 24 00:43:01.626500 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.626415 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9efe9f4c-49de-489f-8fb8-5182ccfa0607" containerName="revoke-keys" Jul 24 00:43:01.626500 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.626488 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9efe9f4c-49de-489f-8fb8-5182ccfa0607" containerName="revoke-keys" Jul 24 00:43:01.628691 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.628674 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" Jul 24 00:43:01.631347 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.631322 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-7f92c6ce\"" Jul 24 00:43:01.631469 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.631322 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-7f92c6ce-dockercfg-r92sf\"" Jul 24 00:43:01.640282 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.640247 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr"] Jul 24 00:43:01.718454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.718418 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wnwt\" (UniqueName: \"kubernetes.io/projected/99e78f85-a296-4c21-95ca-fad0055f415a-kube-api-access-8wnwt\") pod \"payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr\" (UID: \"99e78f85-a296-4c21-95ca-fad0055f415a\") " pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" Jul 24 00:43:01.718868 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.718559 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99e78f85-a296-4c21-95ca-fad0055f415a-plugins-config-volume\") pod \"payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr\" (UID: \"99e78f85-a296-4c21-95ca-fad0055f415a\") " pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" Jul 24 00:43:01.820050 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.820008 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99e78f85-a296-4c21-95ca-fad0055f415a-plugins-config-volume\") pod \"payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr\" (UID: \"99e78f85-a296-4c21-95ca-fad0055f415a\") " pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" Jul 24 00:43:01.820248 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.820061 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wnwt\" (UniqueName: \"kubernetes.io/projected/99e78f85-a296-4c21-95ca-fad0055f415a-kube-api-access-8wnwt\") pod \"payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr\" (UID: \"99e78f85-a296-4c21-95ca-fad0055f415a\") " pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" Jul 24 00:43:01.820767 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.820747 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99e78f85-a296-4c21-95ca-fad0055f415a-plugins-config-volume\") pod \"payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr\" (UID: \"99e78f85-a296-4c21-95ca-fad0055f415a\") " pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" Jul 24 00:43:01.828019 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.827996 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wnwt\" (UniqueName: \"kubernetes.io/projected/99e78f85-a296-4c21-95ca-fad0055f415a-kube-api-access-8wnwt\") pod \"payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr\" (UID: \"99e78f85-a296-4c21-95ca-fad0055f415a\") " pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" Jul 24 00:43:01.941299 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:01.941252 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" Jul 24 00:43:02.083698 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:02.083598 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr"] Jul 24 00:43:02.087096 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:43:02.087071 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99e78f85_a296_4c21_95ca_fad0055f415a.slice/crio-a9fb30a204529f6db9868ee2862d812cc54b26a882ce60ea7e62cad9a27f661f WatchSource:0}: Error finding container a9fb30a204529f6db9868ee2862d812cc54b26a882ce60ea7e62cad9a27f661f: Status 404 returned error can't find the container with id a9fb30a204529f6db9868ee2862d812cc54b26a882ce60ea7e62cad9a27f661f Jul 24 00:43:02.089061 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:02.089040 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 00:43:02.929468 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:02.929388 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" event={"ID":"99e78f85-a296-4c21-95ca-fad0055f415a","Type":"ContainerStarted","Data":"a9fb30a204529f6db9868ee2862d812cc54b26a882ce60ea7e62cad9a27f661f"} Jul 24 00:43:03.935816 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:03.935713 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" event={"ID":"99e78f85-a296-4c21-95ca-fad0055f415a","Type":"ContainerStarted","Data":"45ddf420c66f41430080b5611b84c719a93c7da000983e37166eb6bf5c2507df"} Jul 24 00:43:03.935816 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:03.935803 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" Jul 24 00:43:05.328150 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:05.328118 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:05.893880 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:05.893844 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:05.994352 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:05.994318 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:06.333047 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:06.333012 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:12.153164 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:12.153134 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:12.253487 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:12.253454 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:14.943043 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:14.943008 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" Jul 24 00:43:14.960066 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:14.960010 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" podStartSLOduration=12.516998936 podStartE2EDuration="13.959994924s" totalImagesPullingTime="1.442995988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:43:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:43:03.952167119 +0000 UTC m=+1912.557481427" watchObservedRunningTime="2026-07-24 00:43:14.959994924 +0000 UTC m=+1923.565309239" Jul 24 00:43:15.282036 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:15.282001 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:15.345788 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:15.345753 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:23.070965 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:23.070901 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:23.096979 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:23.096941 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr"] Jul 24 00:43:23.097478 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:23.097443 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" podUID="99e78f85-a296-4c21-95ca-fad0055f415a" containerName="payload-processing" containerID="cri-o://45ddf420c66f41430080b5611b84c719a93c7da000983e37166eb6bf5c2507df" gracePeriod=30 Jul 24 00:43:23.136337 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:23.136285 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:23.237947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:23.237909 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:23.352882 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:23.352848 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" Jul 24 00:43:23.437232 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:23.437189 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wnwt\" (UniqueName: \"kubernetes.io/projected/99e78f85-a296-4c21-95ca-fad0055f415a-kube-api-access-8wnwt\") pod \"99e78f85-a296-4c21-95ca-fad0055f415a\" (UID: \"99e78f85-a296-4c21-95ca-fad0055f415a\") " Jul 24 00:43:23.437232 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:23.437230 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99e78f85-a296-4c21-95ca-fad0055f415a-plugins-config-volume\") pod \"99e78f85-a296-4c21-95ca-fad0055f415a\" (UID: \"99e78f85-a296-4c21-95ca-fad0055f415a\") " Jul 24 00:43:23.437612 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:23.437588 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99e78f85-a296-4c21-95ca-fad0055f415a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "99e78f85-a296-4c21-95ca-fad0055f415a" (UID: "99e78f85-a296-4c21-95ca-fad0055f415a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:43:23.439440 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:23.439417 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99e78f85-a296-4c21-95ca-fad0055f415a-kube-api-access-8wnwt" (OuterVolumeSpecName: "kube-api-access-8wnwt") pod "99e78f85-a296-4c21-95ca-fad0055f415a" (UID: "99e78f85-a296-4c21-95ca-fad0055f415a"). InnerVolumeSpecName "kube-api-access-8wnwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:43:23.538343 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:23.538300 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wnwt\" (UniqueName: \"kubernetes.io/projected/99e78f85-a296-4c21-95ca-fad0055f415a-kube-api-access-8wnwt\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:43:23.538343 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:23.538339 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99e78f85-a296-4c21-95ca-fad0055f415a-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:43:24.042077 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:24.042043 2560 generic.go:356] "Generic (PLEG): container finished" podID="99e78f85-a296-4c21-95ca-fad0055f415a" containerID="45ddf420c66f41430080b5611b84c719a93c7da000983e37166eb6bf5c2507df" exitCode=0 Jul 24 00:43:24.042292 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:24.042089 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" Jul 24 00:43:24.042292 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:24.042088 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" event={"ID":"99e78f85-a296-4c21-95ca-fad0055f415a","Type":"ContainerDied","Data":"45ddf420c66f41430080b5611b84c719a93c7da000983e37166eb6bf5c2507df"} Jul 24 00:43:24.042292 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:24.042229 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr" event={"ID":"99e78f85-a296-4c21-95ca-fad0055f415a","Type":"ContainerDied","Data":"a9fb30a204529f6db9868ee2862d812cc54b26a882ce60ea7e62cad9a27f661f"} Jul 24 00:43:24.042292 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:24.042261 2560 scope.go:117] "RemoveContainer" containerID="45ddf420c66f41430080b5611b84c719a93c7da000983e37166eb6bf5c2507df" Jul 24 00:43:24.051723 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:24.051703 2560 scope.go:117] "RemoveContainer" containerID="45ddf420c66f41430080b5611b84c719a93c7da000983e37166eb6bf5c2507df" Jul 24 00:43:24.052038 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:43:24.052018 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45ddf420c66f41430080b5611b84c719a93c7da000983e37166eb6bf5c2507df\": container with ID starting with 45ddf420c66f41430080b5611b84c719a93c7da000983e37166eb6bf5c2507df not found: ID does not exist" containerID="45ddf420c66f41430080b5611b84c719a93c7da000983e37166eb6bf5c2507df" Jul 24 00:43:24.052098 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:24.052049 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45ddf420c66f41430080b5611b84c719a93c7da000983e37166eb6bf5c2507df"} err="failed to get container status \"45ddf420c66f41430080b5611b84c719a93c7da000983e37166eb6bf5c2507df\": rpc error: code = NotFound desc = could not find container \"45ddf420c66f41430080b5611b84c719a93c7da000983e37166eb6bf5c2507df\": container with ID starting with 45ddf420c66f41430080b5611b84c719a93c7da000983e37166eb6bf5c2507df not found: ID does not exist" Jul 24 00:43:24.062698 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:24.062666 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr"] Jul 24 00:43:24.066877 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:24.066849 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7f92c6ce-5d44f8c474-5rplr"] Jul 24 00:43:24.352701 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:24.352603 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99e78f85-a296-4c21-95ca-fad0055f415a" path="/var/lib/kubelet/pods/99e78f85-a296-4c21-95ca-fad0055f415a/volumes" Jul 24 00:43:27.607971 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:27.607925 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:28.012119 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:28.012065 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:33.726762 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:33.726725 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:33.827514 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:33.827469 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:34.013917 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:34.013883 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:34.233241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:34.233205 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:34.374110 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:34.374027 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:34.478997 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:34.478971 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:34.637620 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:34.637533 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:40.340358 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:40.340322 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:40.445697 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:40.445632 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:40.539340 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:40.539307 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:40.694382 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:40.694293 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:46.329366 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:46.329333 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:46.398722 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:46.398687 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:53.232758 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:53.232725 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:53.621172 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:53.621122 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:53.844985 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:53.844942 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:56.543193 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:56.543139 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:56.622112 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:56.622055 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:56.691262 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:56.691209 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:56.788948 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:56.788906 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:57.204418 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.204361 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:57.300573 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.300526 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:43:57.319585 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.319552 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv"] Jul 24 00:43:57.320020 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.320004 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99e78f85-a296-4c21-95ca-fad0055f415a" containerName="payload-processing" Jul 24 00:43:57.320070 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.320022 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="99e78f85-a296-4c21-95ca-fad0055f415a" containerName="payload-processing" Jul 24 00:43:57.320105 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.320099 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="99e78f85-a296-4c21-95ca-fad0055f415a" containerName="payload-processing" Jul 24 00:43:57.323044 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.323028 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" Jul 24 00:43:57.325574 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.325556 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-464c3345-dockercfg-4nlfr\"" Jul 24 00:43:57.328877 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.328856 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-464c3345\"" Jul 24 00:43:57.335153 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.335129 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv"] Jul 24 00:43:57.351117 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.351076 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777"] Jul 24 00:43:57.354599 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.354574 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" Jul 24 00:43:57.363291 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.363259 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777"] Jul 24 00:43:57.363818 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.363791 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6wjt\" (UniqueName: \"kubernetes.io/projected/746e7d1c-17b2-4192-bdd5-994442e0b289-kube-api-access-t6wjt\") pod \"payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv\" (UID: \"746e7d1c-17b2-4192-bdd5-994442e0b289\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" Jul 24 00:43:57.363885 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.363836 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/746e7d1c-17b2-4192-bdd5-994442e0b289-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv\" (UID: \"746e7d1c-17b2-4192-bdd5-994442e0b289\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" Jul 24 00:43:57.464617 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.464526 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6wjt\" (UniqueName: \"kubernetes.io/projected/746e7d1c-17b2-4192-bdd5-994442e0b289-kube-api-access-t6wjt\") pod \"payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv\" (UID: \"746e7d1c-17b2-4192-bdd5-994442e0b289\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" Jul 24 00:43:57.464617 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.464565 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/746e7d1c-17b2-4192-bdd5-994442e0b289-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv\" (UID: \"746e7d1c-17b2-4192-bdd5-994442e0b289\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" Jul 24 00:43:57.464617 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.464602 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87cdeccb-16df-466d-9a32-3c50ac199dfb-plugins-config-volume\") pod \"payload-processing-e2e-mt-464c3345-56686c4f66-85777\" (UID: \"87cdeccb-16df-466d-9a32-3c50ac199dfb\") " pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" Jul 24 00:43:57.464884 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.464675 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbp6j\" (UniqueName: \"kubernetes.io/projected/87cdeccb-16df-466d-9a32-3c50ac199dfb-kube-api-access-bbp6j\") pod \"payload-processing-e2e-mt-464c3345-56686c4f66-85777\" (UID: \"87cdeccb-16df-466d-9a32-3c50ac199dfb\") " pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" Jul 24 00:43:57.465340 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.465319 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/746e7d1c-17b2-4192-bdd5-994442e0b289-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv\" (UID: \"746e7d1c-17b2-4192-bdd5-994442e0b289\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" Jul 24 00:43:57.473048 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.473027 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6wjt\" (UniqueName: \"kubernetes.io/projected/746e7d1c-17b2-4192-bdd5-994442e0b289-kube-api-access-t6wjt\") pod \"payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv\" (UID: \"746e7d1c-17b2-4192-bdd5-994442e0b289\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" Jul 24 00:43:57.565328 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.565283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbp6j\" (UniqueName: \"kubernetes.io/projected/87cdeccb-16df-466d-9a32-3c50ac199dfb-kube-api-access-bbp6j\") pod \"payload-processing-e2e-mt-464c3345-56686c4f66-85777\" (UID: \"87cdeccb-16df-466d-9a32-3c50ac199dfb\") " pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" Jul 24 00:43:57.565766 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.565368 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87cdeccb-16df-466d-9a32-3c50ac199dfb-plugins-config-volume\") pod \"payload-processing-e2e-mt-464c3345-56686c4f66-85777\" (UID: \"87cdeccb-16df-466d-9a32-3c50ac199dfb\") " pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" Jul 24 00:43:57.566137 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.566116 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87cdeccb-16df-466d-9a32-3c50ac199dfb-plugins-config-volume\") pod \"payload-processing-e2e-mt-464c3345-56686c4f66-85777\" (UID: \"87cdeccb-16df-466d-9a32-3c50ac199dfb\") " pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" Jul 24 00:43:57.573476 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.573446 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbp6j\" (UniqueName: \"kubernetes.io/projected/87cdeccb-16df-466d-9a32-3c50ac199dfb-kube-api-access-bbp6j\") pod \"payload-processing-e2e-mt-464c3345-56686c4f66-85777\" (UID: \"87cdeccb-16df-466d-9a32-3c50ac199dfb\") " pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" Jul 24 00:43:57.634052 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.634009 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" Jul 24 00:43:57.666435 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.666401 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" Jul 24 00:43:57.789099 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.789015 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv"] Jul 24 00:43:57.791508 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:43:57.791479 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod746e7d1c_17b2_4192_bdd5_994442e0b289.slice/crio-b0f1d4d768cd0ad521519626c8c3884a41d68f9dcad29e69a6e2e0ec60a23553 WatchSource:0}: Error finding container b0f1d4d768cd0ad521519626c8c3884a41d68f9dcad29e69a6e2e0ec60a23553: Status 404 returned error can't find the container with id b0f1d4d768cd0ad521519626c8c3884a41d68f9dcad29e69a6e2e0ec60a23553 Jul 24 00:43:57.820197 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:57.820171 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777"] Jul 24 00:43:57.827936 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:43:57.827906 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87cdeccb_16df_466d_9a32_3c50ac199dfb.slice/crio-cf79b4c4ca504fc7e1a9019f66c9df04f254b4a02a33601d5fdfdaeaa5e9309e WatchSource:0}: Error finding container cf79b4c4ca504fc7e1a9019f66c9df04f254b4a02a33601d5fdfdaeaa5e9309e: Status 404 returned error can't find the container with id cf79b4c4ca504fc7e1a9019f66c9df04f254b4a02a33601d5fdfdaeaa5e9309e Jul 24 00:43:58.217819 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:58.217782 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" event={"ID":"87cdeccb-16df-466d-9a32-3c50ac199dfb","Type":"ContainerStarted","Data":"04625c7f75a0ee661f7909e692abafbd62be7caae7e1828b8170a2f5eb99c72d"} Jul 24 00:43:58.218008 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:58.217836 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" event={"ID":"87cdeccb-16df-466d-9a32-3c50ac199dfb","Type":"ContainerStarted","Data":"cf79b4c4ca504fc7e1a9019f66c9df04f254b4a02a33601d5fdfdaeaa5e9309e"} Jul 24 00:43:58.218008 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:58.217900 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" Jul 24 00:43:58.219409 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:58.219383 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" event={"ID":"746e7d1c-17b2-4192-bdd5-994442e0b289","Type":"ContainerStarted","Data":"d8cdb7a7f19975db7004afe3a487eab972ee2f5d9b9dc1d0336300f86f9b99d5"} Jul 24 00:43:58.219484 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:58.219427 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" event={"ID":"746e7d1c-17b2-4192-bdd5-994442e0b289","Type":"ContainerStarted","Data":"b0f1d4d768cd0ad521519626c8c3884a41d68f9dcad29e69a6e2e0ec60a23553"} Jul 24 00:43:58.219484 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:58.219448 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" Jul 24 00:43:58.235867 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:58.235814 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" podStartSLOduration=1.235798811 podStartE2EDuration="1.235798811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:43:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:43:58.233763362 +0000 UTC m=+1966.839077677" watchObservedRunningTime="2026-07-24 00:43:58.235798811 +0000 UTC m=+1966.841113126" Jul 24 00:43:58.250618 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:43:58.250560 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" podStartSLOduration=1.250545721 podStartE2EDuration="1.250545721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:43:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:43:58.24846359 +0000 UTC m=+1966.853777908" watchObservedRunningTime="2026-07-24 00:43:58.250545721 +0000 UTC m=+1966.855860036" Jul 24 00:44:01.037271 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:01.037235 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:44:09.228015 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:09.227977 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" Jul 24 00:44:09.228408 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:09.228051 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" Jul 24 00:44:09.651340 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:09.651268 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:44:17.126247 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:17.126199 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:44:46.099371 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.099339 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:44:46.124526 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.124491 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777"] Jul 24 00:44:46.124785 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.124740 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" podUID="87cdeccb-16df-466d-9a32-3c50ac199dfb" containerName="payload-processing" containerID="cri-o://04625c7f75a0ee661f7909e692abafbd62be7caae7e1828b8170a2f5eb99c72d" gracePeriod=30 Jul 24 00:44:46.132454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.132419 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv"] Jul 24 00:44:46.132697 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.132672 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" podUID="746e7d1c-17b2-4192-bdd5-994442e0b289" containerName="payload-pre-processing" containerID="cri-o://d8cdb7a7f19975db7004afe3a487eab972ee2f5d9b9dc1d0336300f86f9b99d5" gracePeriod=30 Jul 24 00:44:46.164859 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.164825 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:44:46.219754 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.219717 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:44:46.413597 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.413570 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" Jul 24 00:44:46.416472 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.416451 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" Jul 24 00:44:46.473837 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.473800 2560 generic.go:356] "Generic (PLEG): container finished" podID="87cdeccb-16df-466d-9a32-3c50ac199dfb" containerID="04625c7f75a0ee661f7909e692abafbd62be7caae7e1828b8170a2f5eb99c72d" exitCode=0 Jul 24 00:44:46.474027 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.473878 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" Jul 24 00:44:46.474027 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.473891 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" event={"ID":"87cdeccb-16df-466d-9a32-3c50ac199dfb","Type":"ContainerDied","Data":"04625c7f75a0ee661f7909e692abafbd62be7caae7e1828b8170a2f5eb99c72d"} Jul 24 00:44:46.474027 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.473945 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777" event={"ID":"87cdeccb-16df-466d-9a32-3c50ac199dfb","Type":"ContainerDied","Data":"cf79b4c4ca504fc7e1a9019f66c9df04f254b4a02a33601d5fdfdaeaa5e9309e"} Jul 24 00:44:46.474027 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.473975 2560 scope.go:117] "RemoveContainer" containerID="04625c7f75a0ee661f7909e692abafbd62be7caae7e1828b8170a2f5eb99c72d" Jul 24 00:44:46.475281 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.475260 2560 generic.go:356] "Generic (PLEG): container finished" podID="746e7d1c-17b2-4192-bdd5-994442e0b289" containerID="d8cdb7a7f19975db7004afe3a487eab972ee2f5d9b9dc1d0336300f86f9b99d5" exitCode=0 Jul 24 00:44:46.475363 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.475309 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" Jul 24 00:44:46.475363 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.475326 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" event={"ID":"746e7d1c-17b2-4192-bdd5-994442e0b289","Type":"ContainerDied","Data":"d8cdb7a7f19975db7004afe3a487eab972ee2f5d9b9dc1d0336300f86f9b99d5"} Jul 24 00:44:46.475447 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.475362 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv" event={"ID":"746e7d1c-17b2-4192-bdd5-994442e0b289","Type":"ContainerDied","Data":"b0f1d4d768cd0ad521519626c8c3884a41d68f9dcad29e69a6e2e0ec60a23553"} Jul 24 00:44:46.483021 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.482998 2560 scope.go:117] "RemoveContainer" containerID="04625c7f75a0ee661f7909e692abafbd62be7caae7e1828b8170a2f5eb99c72d" Jul 24 00:44:46.483320 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:44:46.483300 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04625c7f75a0ee661f7909e692abafbd62be7caae7e1828b8170a2f5eb99c72d\": container with ID starting with 04625c7f75a0ee661f7909e692abafbd62be7caae7e1828b8170a2f5eb99c72d not found: ID does not exist" containerID="04625c7f75a0ee661f7909e692abafbd62be7caae7e1828b8170a2f5eb99c72d" Jul 24 00:44:46.483380 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.483332 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04625c7f75a0ee661f7909e692abafbd62be7caae7e1828b8170a2f5eb99c72d"} err="failed to get container status \"04625c7f75a0ee661f7909e692abafbd62be7caae7e1828b8170a2f5eb99c72d\": rpc error: code = NotFound desc = could not find container \"04625c7f75a0ee661f7909e692abafbd62be7caae7e1828b8170a2f5eb99c72d\": container with ID starting with 04625c7f75a0ee661f7909e692abafbd62be7caae7e1828b8170a2f5eb99c72d not found: ID does not exist" Jul 24 00:44:46.483380 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.483351 2560 scope.go:117] "RemoveContainer" containerID="d8cdb7a7f19975db7004afe3a487eab972ee2f5d9b9dc1d0336300f86f9b99d5" Jul 24 00:44:46.491629 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.491609 2560 scope.go:117] "RemoveContainer" containerID="d8cdb7a7f19975db7004afe3a487eab972ee2f5d9b9dc1d0336300f86f9b99d5" Jul 24 00:44:46.491956 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:44:46.491936 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8cdb7a7f19975db7004afe3a487eab972ee2f5d9b9dc1d0336300f86f9b99d5\": container with ID starting with d8cdb7a7f19975db7004afe3a487eab972ee2f5d9b9dc1d0336300f86f9b99d5 not found: ID does not exist" containerID="d8cdb7a7f19975db7004afe3a487eab972ee2f5d9b9dc1d0336300f86f9b99d5" Jul 24 00:44:46.492006 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.491965 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8cdb7a7f19975db7004afe3a487eab972ee2f5d9b9dc1d0336300f86f9b99d5"} err="failed to get container status \"d8cdb7a7f19975db7004afe3a487eab972ee2f5d9b9dc1d0336300f86f9b99d5\": rpc error: code = NotFound desc = could not find container \"d8cdb7a7f19975db7004afe3a487eab972ee2f5d9b9dc1d0336300f86f9b99d5\": container with ID starting with d8cdb7a7f19975db7004afe3a487eab972ee2f5d9b9dc1d0336300f86f9b99d5 not found: ID does not exist" Jul 24 00:44:46.548185 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.548144 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/746e7d1c-17b2-4192-bdd5-994442e0b289-plugins-config-volume\") pod \"746e7d1c-17b2-4192-bdd5-994442e0b289\" (UID: \"746e7d1c-17b2-4192-bdd5-994442e0b289\") " Jul 24 00:44:46.548386 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.548247 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbp6j\" (UniqueName: \"kubernetes.io/projected/87cdeccb-16df-466d-9a32-3c50ac199dfb-kube-api-access-bbp6j\") pod \"87cdeccb-16df-466d-9a32-3c50ac199dfb\" (UID: \"87cdeccb-16df-466d-9a32-3c50ac199dfb\") " Jul 24 00:44:46.548386 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.548283 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87cdeccb-16df-466d-9a32-3c50ac199dfb-plugins-config-volume\") pod \"87cdeccb-16df-466d-9a32-3c50ac199dfb\" (UID: \"87cdeccb-16df-466d-9a32-3c50ac199dfb\") " Jul 24 00:44:46.548386 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.548312 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6wjt\" (UniqueName: \"kubernetes.io/projected/746e7d1c-17b2-4192-bdd5-994442e0b289-kube-api-access-t6wjt\") pod \"746e7d1c-17b2-4192-bdd5-994442e0b289\" (UID: \"746e7d1c-17b2-4192-bdd5-994442e0b289\") " Jul 24 00:44:46.548646 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.548614 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/746e7d1c-17b2-4192-bdd5-994442e0b289-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "746e7d1c-17b2-4192-bdd5-994442e0b289" (UID: "746e7d1c-17b2-4192-bdd5-994442e0b289"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:44:46.548776 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.548734 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87cdeccb-16df-466d-9a32-3c50ac199dfb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "87cdeccb-16df-466d-9a32-3c50ac199dfb" (UID: "87cdeccb-16df-466d-9a32-3c50ac199dfb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:44:46.550489 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.550470 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87cdeccb-16df-466d-9a32-3c50ac199dfb-kube-api-access-bbp6j" (OuterVolumeSpecName: "kube-api-access-bbp6j") pod "87cdeccb-16df-466d-9a32-3c50ac199dfb" (UID: "87cdeccb-16df-466d-9a32-3c50ac199dfb"). InnerVolumeSpecName "kube-api-access-bbp6j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:44:46.550561 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.550491 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/746e7d1c-17b2-4192-bdd5-994442e0b289-kube-api-access-t6wjt" (OuterVolumeSpecName: "kube-api-access-t6wjt") pod "746e7d1c-17b2-4192-bdd5-994442e0b289" (UID: "746e7d1c-17b2-4192-bdd5-994442e0b289"). InnerVolumeSpecName "kube-api-access-t6wjt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:44:46.649372 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.649279 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbp6j\" (UniqueName: \"kubernetes.io/projected/87cdeccb-16df-466d-9a32-3c50ac199dfb-kube-api-access-bbp6j\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:44:46.649372 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.649316 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87cdeccb-16df-466d-9a32-3c50ac199dfb-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:44:46.649372 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.649327 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6wjt\" (UniqueName: \"kubernetes.io/projected/746e7d1c-17b2-4192-bdd5-994442e0b289-kube-api-access-t6wjt\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:44:46.649372 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.649336 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/746e7d1c-17b2-4192-bdd5-994442e0b289-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:44:46.799678 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.799577 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777"] Jul 24 00:44:46.801900 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.801864 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-464c3345-56686c4f66-85777"] Jul 24 00:44:46.811547 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.811513 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv"] Jul 24 00:44:46.813255 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:46.813231 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-464c3345-6cd5c95c88-j6czv"] Jul 24 00:44:48.348550 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:48.348517 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="746e7d1c-17b2-4192-bdd5-994442e0b289" path="/var/lib/kubelet/pods/746e7d1c-17b2-4192-bdd5-994442e0b289/volumes" Jul 24 00:44:48.348980 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:48.348921 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87cdeccb-16df-466d-9a32-3c50ac199dfb" path="/var/lib/kubelet/pods/87cdeccb-16df-466d-9a32-3c50ac199dfb/volumes" Jul 24 00:44:50.678709 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:50.678673 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:44:54.105668 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:54.105600 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:44:59.186379 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:44:59.186342 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:00.133635 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:00.133597 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747565-chpbg"] Jul 24 00:45:00.134005 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:00.133992 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87cdeccb-16df-466d-9a32-3c50ac199dfb" containerName="payload-processing" Jul 24 00:45:00.134058 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:00.134006 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="87cdeccb-16df-466d-9a32-3c50ac199dfb" containerName="payload-processing" Jul 24 00:45:00.134058 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:00.134034 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="746e7d1c-17b2-4192-bdd5-994442e0b289" containerName="payload-pre-processing" Jul 24 00:45:00.134058 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:00.134040 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="746e7d1c-17b2-4192-bdd5-994442e0b289" containerName="payload-pre-processing" Jul 24 00:45:00.134157 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:00.134093 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="87cdeccb-16df-466d-9a32-3c50ac199dfb" containerName="payload-processing" Jul 24 00:45:00.134157 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:00.134102 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="746e7d1c-17b2-4192-bdd5-994442e0b289" containerName="payload-pre-processing" Jul 24 00:45:00.138854 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:00.138834 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747565-chpbg" Jul 24 00:45:00.141363 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:00.141332 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-865ss\"" Jul 24 00:45:00.143456 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:00.143424 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747565-chpbg"] Jul 24 00:45:00.151276 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:00.151247 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747565-chpbg" Jul 24 00:45:00.287888 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:00.287856 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747565-chpbg"] Jul 24 00:45:00.291258 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:45:00.291229 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbb7bea3_e6d7_418e_80b8_8867be708972.slice/crio-534f1118434bf391be62dc3b3b8c6055c81c38dfd441161bec6461d1f25ccf43 WatchSource:0}: Error finding container 534f1118434bf391be62dc3b3b8c6055c81c38dfd441161bec6461d1f25ccf43: Status 404 returned error can't find the container with id 534f1118434bf391be62dc3b3b8c6055c81c38dfd441161bec6461d1f25ccf43 Jul 24 00:45:00.551063 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:00.551026 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747565-chpbg" event={"ID":"bbb7bea3-e6d7-418e-80b8-8867be708972","Type":"ContainerStarted","Data":"534f1118434bf391be62dc3b3b8c6055c81c38dfd441161bec6461d1f25ccf43"} Jul 24 00:45:01.558089 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:01.558052 2560 generic.go:356] "Generic (PLEG): container finished" podID="bbb7bea3-e6d7-418e-80b8-8867be708972" containerID="2e73b114da7e1c721f4a0dcd4302ca1ae9e245605460b10de586416964aa0c1c" exitCode=0 Jul 24 00:45:01.558524 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:01.558108 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747565-chpbg" event={"ID":"bbb7bea3-e6d7-418e-80b8-8867be708972","Type":"ContainerDied","Data":"2e73b114da7e1c721f4a0dcd4302ca1ae9e245605460b10de586416964aa0c1c"} Jul 24 00:45:02.697758 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:02.697725 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747565-chpbg" Jul 24 00:45:03.571128 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:03.571097 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747565-chpbg" Jul 24 00:45:03.571311 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:03.571097 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747565-chpbg" event={"ID":"bbb7bea3-e6d7-418e-80b8-8867be708972","Type":"ContainerDied","Data":"534f1118434bf391be62dc3b3b8c6055c81c38dfd441161bec6461d1f25ccf43"} Jul 24 00:45:03.571311 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:03.571220 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="534f1118434bf391be62dc3b3b8c6055c81c38dfd441161bec6461d1f25ccf43" Jul 24 00:45:35.390036 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:35.389944 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:36.354751 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:36.354720 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:41.977326 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:41.977291 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:42.037258 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:42.037191 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:42.258716 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:42.258678 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:42.513243 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:42.513139 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:43.628250 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:43.628219 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:43.727537 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:43.727506 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:43.828529 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:43.828498 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:44.622437 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:44.622399 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:44.723972 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:44.723937 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:44.821571 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:44.821533 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:46.269362 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:46.269325 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:51.999109 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:51.999072 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:52.065576 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:52.065542 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:52.718194 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:52.718157 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:54.029391 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:54.029355 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:59.224952 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:59.224915 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:59.299159 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:59.299125 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:59.438560 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:59.438526 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:45:59.537932 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:45:59.537897 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:09.956429 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:09.956396 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:10.317697 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:10.317666 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:12.299361 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:12.299323 2560 scope.go:117] "RemoveContainer" containerID="26c534f2d321312310f4862e2566383dc0e9e123dbd360fd31f55434fb6b7a4d" Jul 24 00:46:26.569592 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:26.569553 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:31.305728 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:31.305688 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:37.920430 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:37.920391 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:38.020656 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:38.020612 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:38.300595 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:38.300562 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:38.643008 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:38.642922 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:39.016745 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:39.016707 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:41.295838 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:41.295804 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:41.493931 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:41.493894 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:41.750488 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:41.750458 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:41.892508 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:41.892473 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:42.291179 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.291144 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:42.384715 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.384675 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:42.400039 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.400004 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl"] Jul 24 00:46:42.400395 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.400382 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbb7bea3-e6d7-418e-80b8-8867be708972" containerName="cleanup" Jul 24 00:46:42.400454 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.400396 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbb7bea3-e6d7-418e-80b8-8867be708972" containerName="cleanup" Jul 24 00:46:42.400499 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.400476 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbb7bea3-e6d7-418e-80b8-8867be708972" containerName="cleanup" Jul 24 00:46:42.403603 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.403582 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" Jul 24 00:46:42.406620 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.406592 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-c28612-dockercfg-qgshq\"" Jul 24 00:46:42.406765 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.406620 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-c28612\"" Jul 24 00:46:42.416273 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.416244 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl"] Jul 24 00:46:42.419694 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.419664 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5"] Jul 24 00:46:42.423147 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.423126 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" Jul 24 00:46:42.435507 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.435480 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5"] Jul 24 00:46:42.464260 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.464217 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b84739b4-1efc-4f75-beff-3a256fa45df4-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5\" (UID: \"b84739b4-1efc-4f75-beff-3a256fa45df4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" Jul 24 00:46:42.464446 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.464271 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q49q\" (UniqueName: \"kubernetes.io/projected/7cdd7205-507d-4daa-b081-e785472fb1b8-kube-api-access-4q49q\") pod \"payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl\" (UID: \"7cdd7205-507d-4daa-b081-e785472fb1b8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" Jul 24 00:46:42.464446 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.464329 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cdd7205-507d-4daa-b081-e785472fb1b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl\" (UID: \"7cdd7205-507d-4daa-b081-e785472fb1b8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" Jul 24 00:46:42.464446 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.464417 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j6df\" (UniqueName: \"kubernetes.io/projected/b84739b4-1efc-4f75-beff-3a256fa45df4-kube-api-access-4j6df\") pod \"payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5\" (UID: \"b84739b4-1efc-4f75-beff-3a256fa45df4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" Jul 24 00:46:42.565917 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.565814 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j6df\" (UniqueName: \"kubernetes.io/projected/b84739b4-1efc-4f75-beff-3a256fa45df4-kube-api-access-4j6df\") pod \"payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5\" (UID: \"b84739b4-1efc-4f75-beff-3a256fa45df4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" Jul 24 00:46:42.565917 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.565878 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b84739b4-1efc-4f75-beff-3a256fa45df4-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5\" (UID: \"b84739b4-1efc-4f75-beff-3a256fa45df4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" Jul 24 00:46:42.566117 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.565927 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q49q\" (UniqueName: \"kubernetes.io/projected/7cdd7205-507d-4daa-b081-e785472fb1b8-kube-api-access-4q49q\") pod \"payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl\" (UID: \"7cdd7205-507d-4daa-b081-e785472fb1b8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" Jul 24 00:46:42.566117 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.565962 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cdd7205-507d-4daa-b081-e785472fb1b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl\" (UID: \"7cdd7205-507d-4daa-b081-e785472fb1b8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" Jul 24 00:46:42.566709 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.566687 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b84739b4-1efc-4f75-beff-3a256fa45df4-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5\" (UID: \"b84739b4-1efc-4f75-beff-3a256fa45df4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" Jul 24 00:46:42.566922 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.566904 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cdd7205-507d-4daa-b081-e785472fb1b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl\" (UID: \"7cdd7205-507d-4daa-b081-e785472fb1b8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" Jul 24 00:46:42.574675 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.574629 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q49q\" (UniqueName: \"kubernetes.io/projected/7cdd7205-507d-4daa-b081-e785472fb1b8-kube-api-access-4q49q\") pod \"payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl\" (UID: \"7cdd7205-507d-4daa-b081-e785472fb1b8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" Jul 24 00:46:42.574797 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.574680 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j6df\" (UniqueName: \"kubernetes.io/projected/b84739b4-1efc-4f75-beff-3a256fa45df4-kube-api-access-4j6df\") pod \"payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5\" (UID: \"b84739b4-1efc-4f75-beff-3a256fa45df4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" Jul 24 00:46:42.715348 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.715306 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" Jul 24 00:46:42.735115 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.735083 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" Jul 24 00:46:42.866849 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.866586 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl"] Jul 24 00:46:42.869729 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:46:42.869692 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cdd7205_507d_4daa_b081_e785472fb1b8.slice/crio-05f38f71e11990cd8661f89e88c00fc3a4fcba74e930d8625682cfd3141848a0 WatchSource:0}: Error finding container 05f38f71e11990cd8661f89e88c00fc3a4fcba74e930d8625682cfd3141848a0: Status 404 returned error can't find the container with id 05f38f71e11990cd8661f89e88c00fc3a4fcba74e930d8625682cfd3141848a0 Jul 24 00:46:42.897541 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:42.897514 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5"] Jul 24 00:46:42.904844 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:46:42.904816 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb84739b4_1efc_4f75_beff_3a256fa45df4.slice/crio-ac728d307d377bef89ebc20d2f4c60567e8b03a386a6ecb67a1ab8242ff8a8c9 WatchSource:0}: Error finding container ac728d307d377bef89ebc20d2f4c60567e8b03a386a6ecb67a1ab8242ff8a8c9: Status 404 returned error can't find the container with id ac728d307d377bef89ebc20d2f4c60567e8b03a386a6ecb67a1ab8242ff8a8c9 Jul 24 00:46:43.095048 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:43.094943 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" event={"ID":"b84739b4-1efc-4f75-beff-3a256fa45df4","Type":"ContainerStarted","Data":"4489ddf0a12f50121cbe0e70b88d17a57268d6f5b6a48177b2ca0c2540498b5f"} Jul 24 00:46:43.095048 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:43.095013 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" event={"ID":"b84739b4-1efc-4f75-beff-3a256fa45df4","Type":"ContainerStarted","Data":"ac728d307d377bef89ebc20d2f4c60567e8b03a386a6ecb67a1ab8242ff8a8c9"} Jul 24 00:46:43.095048 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:43.095042 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" Jul 24 00:46:43.096498 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:43.096471 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" event={"ID":"7cdd7205-507d-4daa-b081-e785472fb1b8","Type":"ContainerStarted","Data":"00b0945d94e67fecb8afe1017ac571bf7a16887484ff21f0a7aa3522dfd74e5d"} Jul 24 00:46:43.096606 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:43.096517 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" event={"ID":"7cdd7205-507d-4daa-b081-e785472fb1b8","Type":"ContainerStarted","Data":"05f38f71e11990cd8661f89e88c00fc3a4fcba74e930d8625682cfd3141848a0"} Jul 24 00:46:43.096606 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:43.096537 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" Jul 24 00:46:43.128816 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:43.128760 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" podStartSLOduration=1.12874269 podStartE2EDuration="1.12874269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:46:43.126626127 +0000 UTC m=+2131.731940445" watchObservedRunningTime="2026-07-24 00:46:43.12874269 +0000 UTC m=+2131.734057006" Jul 24 00:46:43.162995 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:43.162936 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" podStartSLOduration=1.162917416 podStartE2EDuration="1.162917416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:46:43.160553369 +0000 UTC m=+2131.765867686" watchObservedRunningTime="2026-07-24 00:46:43.162917416 +0000 UTC m=+2131.768231732" Jul 24 00:46:45.725404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:45.725371 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:46.087455 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.087413 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:46.405737 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.405630 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm"] Jul 24 00:46:46.409192 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.409175 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.422436 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.422410 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-5f040e-openshift-default-dockercfg-zz2f8\"" Jul 24 00:46:46.440924 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.440892 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm"] Jul 24 00:46:46.480419 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.480383 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:46.504793 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.504747 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-workload-socket\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.504974 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.504824 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-data\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.504974 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.504917 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-credential-socket\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.505112 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.504999 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8zm7\" (UniqueName: \"kubernetes.io/projected/94d34c32-cf1b-4de9-b07f-5ba888b14f00-kube-api-access-s8zm7\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.505112 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.505034 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-podinfo\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.505213 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.505108 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-envoy\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.505213 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.505162 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-workload-certs\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.505213 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.505199 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istiod-ca-cert\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.505364 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.505232 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-token\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.606601 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.606566 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-envoy\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.606801 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.606626 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-workload-certs\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.606801 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.606678 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istiod-ca-cert\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.606801 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.606706 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-token\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.606801 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.606751 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-workload-socket\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.606801 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.606778 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-data\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.607075 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.606812 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-credential-socket\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.607075 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.606875 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8zm7\" (UniqueName: \"kubernetes.io/projected/94d34c32-cf1b-4de9-b07f-5ba888b14f00-kube-api-access-s8zm7\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.607075 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.607049 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-podinfo\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.607260 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.607193 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-workload-socket\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.607260 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.607245 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-data\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.607364 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.607306 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-credential-socket\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.607364 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.607324 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-workload-certs\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.607462 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.607439 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istiod-ca-cert\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.609039 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.609015 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-envoy\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.609387 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.609368 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-podinfo\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.615234 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.615211 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-token\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.615609 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.615588 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8zm7\" (UniqueName: \"kubernetes.io/projected/94d34c32-cf1b-4de9-b07f-5ba888b14f00-kube-api-access-s8zm7\") pod \"e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.650900 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.650868 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:46.720216 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.720124 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:46.866223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.866196 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm"] Jul 24 00:46:46.867968 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:46:46.867940 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94d34c32_cf1b_4de9_b07f_5ba888b14f00.slice/crio-bff59756af62cfa5a3fa68dff2c0de7d21264a74f83fb4bc25c6c883efa527fa WatchSource:0}: Error finding container bff59756af62cfa5a3fa68dff2c0de7d21264a74f83fb4bc25c6c883efa527fa: Status 404 returned error can't find the container with id bff59756af62cfa5a3fa68dff2c0de7d21264a74f83fb4bc25c6c883efa527fa Jul 24 00:46:46.870293 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.870261 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:46:46.870400 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.870326 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:46:46.870400 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:46.870354 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:46:47.119334 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:47.119297 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" event={"ID":"94d34c32-cf1b-4de9-b07f-5ba888b14f00","Type":"ContainerStarted","Data":"1a2602a72a1b09932f040e9d6d0c39c3d1bfbdb393262c0ad9bbf49eccb49504"} Jul 24 00:46:47.119541 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:47.119356 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" event={"ID":"94d34c32-cf1b-4de9-b07f-5ba888b14f00","Type":"ContainerStarted","Data":"bff59756af62cfa5a3fa68dff2c0de7d21264a74f83fb4bc25c6c883efa527fa"} Jul 24 00:46:47.143654 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:47.143593 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" podStartSLOduration=1.143573808 podStartE2EDuration="1.143573808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:46:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:46:47.141248789 +0000 UTC m=+2135.746563107" watchObservedRunningTime="2026-07-24 00:46:47.143573808 +0000 UTC m=+2135.748888125" Jul 24 00:46:47.720692 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:47.720630 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:47.725616 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:47.725584 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:48.124782 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:48.124748 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:48.125823 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:48.125799 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:46:49.212607 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:49.212565 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:49.317028 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:49.316993 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:49.377696 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:49.377664 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:49.754196 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:49.754160 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:49.900694 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:49.900625 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:50.003094 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.003052 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk"] Jul 24 00:46:50.004861 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.004798 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:50.006893 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.006874 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" Jul 24 00:46:50.009371 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.009352 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-5f040e-dockercfg-m6w9l\"" Jul 24 00:46:50.009478 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.009392 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-5f040e\"" Jul 24 00:46:50.017161 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.017136 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk"] Jul 24 00:46:50.041081 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.041044 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5"] Jul 24 00:46:50.044564 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.044545 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" Jul 24 00:46:50.055259 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.055232 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5"] Jul 24 00:46:50.141310 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.141271 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdcgl\" (UniqueName: \"kubernetes.io/projected/a7421728-99ca-4881-8c76-755fcad216fb-kube-api-access-cdcgl\") pod \"payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk\" (UID: \"a7421728-99ca-4881-8c76-755fcad216fb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" Jul 24 00:46:50.141481 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.141342 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/346bb733-1a4a-4359-bbdd-86af8235f1bc-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5\" (UID: \"346bb733-1a4a-4359-bbdd-86af8235f1bc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" Jul 24 00:46:50.141481 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.141440 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb6sr\" (UniqueName: \"kubernetes.io/projected/346bb733-1a4a-4359-bbdd-86af8235f1bc-kube-api-access-lb6sr\") pod \"payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5\" (UID: \"346bb733-1a4a-4359-bbdd-86af8235f1bc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" Jul 24 00:46:50.141573 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.141490 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7421728-99ca-4881-8c76-755fcad216fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk\" (UID: \"a7421728-99ca-4881-8c76-755fcad216fb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" Jul 24 00:46:50.242482 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.242436 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb6sr\" (UniqueName: \"kubernetes.io/projected/346bb733-1a4a-4359-bbdd-86af8235f1bc-kube-api-access-lb6sr\") pod \"payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5\" (UID: \"346bb733-1a4a-4359-bbdd-86af8235f1bc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" Jul 24 00:46:50.243021 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.242517 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7421728-99ca-4881-8c76-755fcad216fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk\" (UID: \"a7421728-99ca-4881-8c76-755fcad216fb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" Jul 24 00:46:50.243021 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.242553 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdcgl\" (UniqueName: \"kubernetes.io/projected/a7421728-99ca-4881-8c76-755fcad216fb-kube-api-access-cdcgl\") pod \"payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk\" (UID: \"a7421728-99ca-4881-8c76-755fcad216fb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" Jul 24 00:46:50.243021 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.242784 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/346bb733-1a4a-4359-bbdd-86af8235f1bc-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5\" (UID: \"346bb733-1a4a-4359-bbdd-86af8235f1bc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" Jul 24 00:46:50.243384 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.243358 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7421728-99ca-4881-8c76-755fcad216fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk\" (UID: \"a7421728-99ca-4881-8c76-755fcad216fb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" Jul 24 00:46:50.243567 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.243546 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/346bb733-1a4a-4359-bbdd-86af8235f1bc-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5\" (UID: \"346bb733-1a4a-4359-bbdd-86af8235f1bc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" Jul 24 00:46:50.250582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.250557 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb6sr\" (UniqueName: \"kubernetes.io/projected/346bb733-1a4a-4359-bbdd-86af8235f1bc-kube-api-access-lb6sr\") pod \"payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5\" (UID: \"346bb733-1a4a-4359-bbdd-86af8235f1bc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" Jul 24 00:46:50.250745 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.250566 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdcgl\" (UniqueName: \"kubernetes.io/projected/a7421728-99ca-4881-8c76-755fcad216fb-kube-api-access-cdcgl\") pod \"payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk\" (UID: \"a7421728-99ca-4881-8c76-755fcad216fb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" Jul 24 00:46:50.319214 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.319124 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" Jul 24 00:46:50.356355 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.356321 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" Jul 24 00:46:50.475447 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.475342 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk"] Jul 24 00:46:50.475777 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:46:50.475749 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7421728_99ca_4881_8c76_755fcad216fb.slice/crio-364aaa80c91efd37175ba828aff679fef6745aa9c7255cd3b8fe7ba64aaf0245 WatchSource:0}: Error finding container 364aaa80c91efd37175ba828aff679fef6745aa9c7255cd3b8fe7ba64aaf0245: Status 404 returned error can't find the container with id 364aaa80c91efd37175ba828aff679fef6745aa9c7255cd3b8fe7ba64aaf0245 Jul 24 00:46:50.512335 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:50.512307 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5"] Jul 24 00:46:50.514863 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:46:50.514837 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod346bb733_1a4a_4359_bbdd_86af8235f1bc.slice/crio-8467803f528e1e3635761a405646a9146c26bd145de1c1ba234bd1242e1d2b70 WatchSource:0}: Error finding container 8467803f528e1e3635761a405646a9146c26bd145de1c1ba234bd1242e1d2b70: Status 404 returned error can't find the container with id 8467803f528e1e3635761a405646a9146c26bd145de1c1ba234bd1242e1d2b70 Jul 24 00:46:51.091802 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:51.091767 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:51.141013 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:51.140979 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" event={"ID":"346bb733-1a4a-4359-bbdd-86af8235f1bc","Type":"ContainerStarted","Data":"6373e0b836b5f42ed27f57ff3bfc313f746ed039af3c1fc94c58bafe45c3f1f4"} Jul 24 00:46:51.141199 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:51.141040 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" Jul 24 00:46:51.141199 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:51.141067 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" event={"ID":"346bb733-1a4a-4359-bbdd-86af8235f1bc","Type":"ContainerStarted","Data":"8467803f528e1e3635761a405646a9146c26bd145de1c1ba234bd1242e1d2b70"} Jul 24 00:46:51.142565 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:51.142538 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" event={"ID":"a7421728-99ca-4881-8c76-755fcad216fb","Type":"ContainerStarted","Data":"01b9ba4f73d64e2f3c5e9a274ed66c8b18dd492d56551f6ab48e63fe588cde96"} Jul 24 00:46:51.142708 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:51.142587 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" event={"ID":"a7421728-99ca-4881-8c76-755fcad216fb","Type":"ContainerStarted","Data":"364aaa80c91efd37175ba828aff679fef6745aa9c7255cd3b8fe7ba64aaf0245"} Jul 24 00:46:51.142708 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:51.142609 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" Jul 24 00:46:51.159086 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:51.159035 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" podStartSLOduration=1.1590188 podStartE2EDuration="1.1590188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:46:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:46:51.157931743 +0000 UTC m=+2139.763246061" watchObservedRunningTime="2026-07-24 00:46:51.1590188 +0000 UTC m=+2139.764333116" Jul 24 00:46:51.176561 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:51.176497 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" podStartSLOduration=2.176474063 podStartE2EDuration="2.176474063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:46:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:46:51.173393545 +0000 UTC m=+2139.778707874" watchObservedRunningTime="2026-07-24 00:46:51.176474063 +0000 UTC m=+2139.781788380" Jul 24 00:46:53.644993 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:53.644956 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:46:54.104252 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:54.104218 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" Jul 24 00:46:54.104434 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:54.104296 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" Jul 24 00:46:59.627047 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:46:59.627011 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:47:00.190734 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:00.190698 2560 generic.go:356] "Generic (PLEG): container finished" podID="94d34c32-cf1b-4de9-b07f-5ba888b14f00" containerID="1a2602a72a1b09932f040e9d6d0c39c3d1bfbdb393262c0ad9bbf49eccb49504" exitCode=0 Jul 24 00:47:00.190951 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:00.190751 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" event={"ID":"94d34c32-cf1b-4de9-b07f-5ba888b14f00","Type":"ContainerDied","Data":"1a2602a72a1b09932f040e9d6d0c39c3d1bfbdb393262c0ad9bbf49eccb49504"} Jul 24 00:47:00.191228 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:00.191212 2560 scope.go:117] "RemoveContainer" containerID="1a2602a72a1b09932f040e9d6d0c39c3d1bfbdb393262c0ad9bbf49eccb49504" Jul 24 00:47:00.192355 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:00.192313 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:47:00.192471 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:00.192420 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:47:00.192536 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:00.192470 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:47:01.199902 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:01.199863 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" event={"ID":"94d34c32-cf1b-4de9-b07f-5ba888b14f00","Type":"ContainerStarted","Data":"4df08d9f2161e45ff555cb322dbf4f942bf88dece2b09f813e710a0b7253d52d"} Jul 24 00:47:01.721029 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:01.720986 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:47:01.722340 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:01.722314 2560 patch_prober.go:28] interesting pod/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 24 00:47:01.722453 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:01.722364 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" podUID="94d34c32-cf1b-4de9-b07f-5ba888b14f00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 24 00:47:02.150473 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:02.150439 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" Jul 24 00:47:02.150662 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:02.150601 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" Jul 24 00:47:02.720776 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:02.720739 2560 patch_prober.go:28] interesting pod/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 24 00:47:02.721250 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:02.720805 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" podUID="94d34c32-cf1b-4de9-b07f-5ba888b14f00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 24 00:47:03.125712 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:03.125600 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:47:03.721506 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:03.721466 2560 patch_prober.go:28] interesting pod/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 24 00:47:03.721950 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:03.721539 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" podUID="94d34c32-cf1b-4de9-b07f-5ba888b14f00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 24 00:47:04.720592 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:04.720552 2560 patch_prober.go:28] interesting pod/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 24 00:47:04.720815 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:04.720625 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" podUID="94d34c32-cf1b-4de9-b07f-5ba888b14f00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 24 00:47:05.725372 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:05.725337 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:47:05.726374 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:05.726353 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:47:12.320952 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:12.320920 2560 scope.go:117] "RemoveContainer" containerID="c42887026df2e9098c96a512c7da2cbff1e2dd5fe81a0f071ba06d7d3d31f6d0" Jul 24 00:47:12.329829 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:12.329808 2560 scope.go:117] "RemoveContainer" containerID="3c3dfd95e0173b86d65a16f5b462022523a61522bf56881d1a36e66bb37290cc" Jul 24 00:47:28.228478 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:47:28.228446 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:08.286773 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:08.286737 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:09.461217 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:09.461186 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:19.922055 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:19.922019 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:19.956583 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:19.956538 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5"] Jul 24 00:48:19.959921 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:19.959861 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" podUID="b84739b4-1efc-4f75-beff-3a256fa45df4" containerName="payload-processing" containerID="cri-o://4489ddf0a12f50121cbe0e70b88d17a57268d6f5b6a48177b2ca0c2540498b5f" gracePeriod=30 Jul 24 00:48:19.962096 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:19.962066 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl"] Jul 24 00:48:19.962521 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:19.962493 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" podUID="7cdd7205-507d-4daa-b081-e785472fb1b8" containerName="payload-pre-processing" containerID="cri-o://00b0945d94e67fecb8afe1017ac571bf7a16887484ff21f0a7aa3522dfd74e5d" gracePeriod=30 Jul 24 00:48:20.005096 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.005045 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:20.082491 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.082454 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:20.278109 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.278084 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" Jul 24 00:48:20.281154 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.281131 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" Jul 24 00:48:20.448709 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.448586 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cdd7205-507d-4daa-b081-e785472fb1b8-plugins-config-volume\") pod \"7cdd7205-507d-4daa-b081-e785472fb1b8\" (UID: \"7cdd7205-507d-4daa-b081-e785472fb1b8\") " Jul 24 00:48:20.448875 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.448728 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b84739b4-1efc-4f75-beff-3a256fa45df4-plugins-config-volume\") pod \"b84739b4-1efc-4f75-beff-3a256fa45df4\" (UID: \"b84739b4-1efc-4f75-beff-3a256fa45df4\") " Jul 24 00:48:20.448875 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.448762 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4q49q\" (UniqueName: \"kubernetes.io/projected/7cdd7205-507d-4daa-b081-e785472fb1b8-kube-api-access-4q49q\") pod \"7cdd7205-507d-4daa-b081-e785472fb1b8\" (UID: \"7cdd7205-507d-4daa-b081-e785472fb1b8\") " Jul 24 00:48:20.448875 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.448800 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4j6df\" (UniqueName: \"kubernetes.io/projected/b84739b4-1efc-4f75-beff-3a256fa45df4-kube-api-access-4j6df\") pod \"b84739b4-1efc-4f75-beff-3a256fa45df4\" (UID: \"b84739b4-1efc-4f75-beff-3a256fa45df4\") " Jul 24 00:48:20.449109 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.449073 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cdd7205-507d-4daa-b081-e785472fb1b8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7cdd7205-507d-4daa-b081-e785472fb1b8" (UID: "7cdd7205-507d-4daa-b081-e785472fb1b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:48:20.449239 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.449117 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b84739b4-1efc-4f75-beff-3a256fa45df4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b84739b4-1efc-4f75-beff-3a256fa45df4" (UID: "b84739b4-1efc-4f75-beff-3a256fa45df4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:48:20.449511 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.449477 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b84739b4-1efc-4f75-beff-3a256fa45df4-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:20.449617 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.449568 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cdd7205-507d-4daa-b081-e785472fb1b8-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:20.451030 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.451008 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cdd7205-507d-4daa-b081-e785472fb1b8-kube-api-access-4q49q" (OuterVolumeSpecName: "kube-api-access-4q49q") pod "7cdd7205-507d-4daa-b081-e785472fb1b8" (UID: "7cdd7205-507d-4daa-b081-e785472fb1b8"). InnerVolumeSpecName "kube-api-access-4q49q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:48:20.451030 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.451019 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b84739b4-1efc-4f75-beff-3a256fa45df4-kube-api-access-4j6df" (OuterVolumeSpecName: "kube-api-access-4j6df") pod "b84739b4-1efc-4f75-beff-3a256fa45df4" (UID: "b84739b4-1efc-4f75-beff-3a256fa45df4"). InnerVolumeSpecName "kube-api-access-4j6df". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:48:20.551039 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.550999 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q49q\" (UniqueName: \"kubernetes.io/projected/7cdd7205-507d-4daa-b081-e785472fb1b8-kube-api-access-4q49q\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:20.551039 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.551036 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4j6df\" (UniqueName: \"kubernetes.io/projected/b84739b4-1efc-4f75-beff-3a256fa45df4-kube-api-access-4j6df\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:20.630944 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.630911 2560 generic.go:356] "Generic (PLEG): container finished" podID="b84739b4-1efc-4f75-beff-3a256fa45df4" containerID="4489ddf0a12f50121cbe0e70b88d17a57268d6f5b6a48177b2ca0c2540498b5f" exitCode=0 Jul 24 00:48:20.631123 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.630971 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" Jul 24 00:48:20.631123 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.630986 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" event={"ID":"b84739b4-1efc-4f75-beff-3a256fa45df4","Type":"ContainerDied","Data":"4489ddf0a12f50121cbe0e70b88d17a57268d6f5b6a48177b2ca0c2540498b5f"} Jul 24 00:48:20.631123 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.631036 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5" event={"ID":"b84739b4-1efc-4f75-beff-3a256fa45df4","Type":"ContainerDied","Data":"ac728d307d377bef89ebc20d2f4c60567e8b03a386a6ecb67a1ab8242ff8a8c9"} Jul 24 00:48:20.631123 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.631058 2560 scope.go:117] "RemoveContainer" containerID="4489ddf0a12f50121cbe0e70b88d17a57268d6f5b6a48177b2ca0c2540498b5f" Jul 24 00:48:20.632295 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.632276 2560 generic.go:356] "Generic (PLEG): container finished" podID="7cdd7205-507d-4daa-b081-e785472fb1b8" containerID="00b0945d94e67fecb8afe1017ac571bf7a16887484ff21f0a7aa3522dfd74e5d" exitCode=0 Jul 24 00:48:20.632384 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.632335 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" event={"ID":"7cdd7205-507d-4daa-b081-e785472fb1b8","Type":"ContainerDied","Data":"00b0945d94e67fecb8afe1017ac571bf7a16887484ff21f0a7aa3522dfd74e5d"} Jul 24 00:48:20.632384 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.632366 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" event={"ID":"7cdd7205-507d-4daa-b081-e785472fb1b8","Type":"ContainerDied","Data":"05f38f71e11990cd8661f89e88c00fc3a4fcba74e930d8625682cfd3141848a0"} Jul 24 00:48:20.632498 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.632424 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl" Jul 24 00:48:20.641208 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.641113 2560 scope.go:117] "RemoveContainer" containerID="4489ddf0a12f50121cbe0e70b88d17a57268d6f5b6a48177b2ca0c2540498b5f" Jul 24 00:48:20.641500 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:48:20.641468 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4489ddf0a12f50121cbe0e70b88d17a57268d6f5b6a48177b2ca0c2540498b5f\": container with ID starting with 4489ddf0a12f50121cbe0e70b88d17a57268d6f5b6a48177b2ca0c2540498b5f not found: ID does not exist" containerID="4489ddf0a12f50121cbe0e70b88d17a57268d6f5b6a48177b2ca0c2540498b5f" Jul 24 00:48:20.641624 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.641512 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4489ddf0a12f50121cbe0e70b88d17a57268d6f5b6a48177b2ca0c2540498b5f"} err="failed to get container status \"4489ddf0a12f50121cbe0e70b88d17a57268d6f5b6a48177b2ca0c2540498b5f\": rpc error: code = NotFound desc = could not find container \"4489ddf0a12f50121cbe0e70b88d17a57268d6f5b6a48177b2ca0c2540498b5f\": container with ID starting with 4489ddf0a12f50121cbe0e70b88d17a57268d6f5b6a48177b2ca0c2540498b5f not found: ID does not exist" Jul 24 00:48:20.641624 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.641534 2560 scope.go:117] "RemoveContainer" containerID="00b0945d94e67fecb8afe1017ac571bf7a16887484ff21f0a7aa3522dfd74e5d" Jul 24 00:48:20.649918 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.649899 2560 scope.go:117] "RemoveContainer" containerID="00b0945d94e67fecb8afe1017ac571bf7a16887484ff21f0a7aa3522dfd74e5d" Jul 24 00:48:20.650178 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:48:20.650159 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00b0945d94e67fecb8afe1017ac571bf7a16887484ff21f0a7aa3522dfd74e5d\": container with ID starting with 00b0945d94e67fecb8afe1017ac571bf7a16887484ff21f0a7aa3522dfd74e5d not found: ID does not exist" containerID="00b0945d94e67fecb8afe1017ac571bf7a16887484ff21f0a7aa3522dfd74e5d" Jul 24 00:48:20.650245 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.650196 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00b0945d94e67fecb8afe1017ac571bf7a16887484ff21f0a7aa3522dfd74e5d"} err="failed to get container status \"00b0945d94e67fecb8afe1017ac571bf7a16887484ff21f0a7aa3522dfd74e5d\": rpc error: code = NotFound desc = could not find container \"00b0945d94e67fecb8afe1017ac571bf7a16887484ff21f0a7aa3522dfd74e5d\": container with ID starting with 00b0945d94e67fecb8afe1017ac571bf7a16887484ff21f0a7aa3522dfd74e5d not found: ID does not exist" Jul 24 00:48:20.659546 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.659518 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5"] Jul 24 00:48:20.663547 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.663514 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c28612-7bfc478f4b-c4xs5"] Jul 24 00:48:20.673574 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.673543 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl"] Jul 24 00:48:20.677861 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:20.677837 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c28612-7f779785fb-djtzl"] Jul 24 00:48:22.349982 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:22.349949 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cdd7205-507d-4daa-b081-e785472fb1b8" path="/var/lib/kubelet/pods/7cdd7205-507d-4daa-b081-e785472fb1b8/volumes" Jul 24 00:48:22.350512 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:22.350491 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b84739b4-1efc-4f75-beff-3a256fa45df4" path="/var/lib/kubelet/pods/b84739b4-1efc-4f75-beff-3a256fa45df4/volumes" Jul 24 00:48:24.477025 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:24.476982 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:24.938403 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:24.938357 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:30.850448 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:30.850414 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:30.923951 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:30.923916 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:31.132187 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:31.132085 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:31.358660 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:31.358594 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:42.098537 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.098451 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:42.124202 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.124167 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5"] Jul 24 00:48:42.124521 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.124468 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" podUID="346bb733-1a4a-4359-bbdd-86af8235f1bc" containerName="payload-processing" containerID="cri-o://6373e0b836b5f42ed27f57ff3bfc313f746ed039af3c1fc94c58bafe45c3f1f4" gracePeriod=30 Jul 24 00:48:42.131974 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.131944 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk"] Jul 24 00:48:42.132199 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.132170 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" podUID="a7421728-99ca-4881-8c76-755fcad216fb" containerName="payload-pre-processing" containerID="cri-o://01b9ba4f73d64e2f3c5e9a274ed66c8b18dd492d56551f6ab48e63fe588cde96" gracePeriod=30 Jul 24 00:48:42.149174 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.149047 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" podUID="a7421728-99ca-4881-8c76-755fcad216fb" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.48:9004: connect: connection refused" Jul 24 00:48:42.149174 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.149077 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" podUID="346bb733-1a4a-4359-bbdd-86af8235f1bc" containerName="payload-processing" probeResult="failure" output="dial tcp 10.134.0.49:9004: connect: connection refused" Jul 24 00:48:42.197065 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.197019 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:42.297784 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.297632 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:42.395818 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.395792 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" Jul 24 00:48:42.420162 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.420128 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" Jul 24 00:48:42.470046 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.470004 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lb6sr\" (UniqueName: \"kubernetes.io/projected/346bb733-1a4a-4359-bbdd-86af8235f1bc-kube-api-access-lb6sr\") pod \"346bb733-1a4a-4359-bbdd-86af8235f1bc\" (UID: \"346bb733-1a4a-4359-bbdd-86af8235f1bc\") " Jul 24 00:48:42.470231 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.470141 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/346bb733-1a4a-4359-bbdd-86af8235f1bc-plugins-config-volume\") pod \"346bb733-1a4a-4359-bbdd-86af8235f1bc\" (UID: \"346bb733-1a4a-4359-bbdd-86af8235f1bc\") " Jul 24 00:48:42.470511 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.470484 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/346bb733-1a4a-4359-bbdd-86af8235f1bc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "346bb733-1a4a-4359-bbdd-86af8235f1bc" (UID: "346bb733-1a4a-4359-bbdd-86af8235f1bc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:48:42.472105 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.472084 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/346bb733-1a4a-4359-bbdd-86af8235f1bc-kube-api-access-lb6sr" (OuterVolumeSpecName: "kube-api-access-lb6sr") pod "346bb733-1a4a-4359-bbdd-86af8235f1bc" (UID: "346bb733-1a4a-4359-bbdd-86af8235f1bc"). InnerVolumeSpecName "kube-api-access-lb6sr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:48:42.570985 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.570944 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7421728-99ca-4881-8c76-755fcad216fb-plugins-config-volume\") pod \"a7421728-99ca-4881-8c76-755fcad216fb\" (UID: \"a7421728-99ca-4881-8c76-755fcad216fb\") " Jul 24 00:48:42.571203 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.571039 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdcgl\" (UniqueName: \"kubernetes.io/projected/a7421728-99ca-4881-8c76-755fcad216fb-kube-api-access-cdcgl\") pod \"a7421728-99ca-4881-8c76-755fcad216fb\" (UID: \"a7421728-99ca-4881-8c76-755fcad216fb\") " Jul 24 00:48:42.571279 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.571249 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/346bb733-1a4a-4359-bbdd-86af8235f1bc-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:42.571279 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.571261 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb6sr\" (UniqueName: \"kubernetes.io/projected/346bb733-1a4a-4359-bbdd-86af8235f1bc-kube-api-access-lb6sr\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:42.571444 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.571417 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7421728-99ca-4881-8c76-755fcad216fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7421728-99ca-4881-8c76-755fcad216fb" (UID: "a7421728-99ca-4881-8c76-755fcad216fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:48:42.573172 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.573147 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7421728-99ca-4881-8c76-755fcad216fb-kube-api-access-cdcgl" (OuterVolumeSpecName: "kube-api-access-cdcgl") pod "a7421728-99ca-4881-8c76-755fcad216fb" (UID: "a7421728-99ca-4881-8c76-755fcad216fb"). InnerVolumeSpecName "kube-api-access-cdcgl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:48:42.672528 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.672439 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdcgl\" (UniqueName: \"kubernetes.io/projected/a7421728-99ca-4881-8c76-755fcad216fb-kube-api-access-cdcgl\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:42.672528 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.672471 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7421728-99ca-4881-8c76-755fcad216fb-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:42.757607 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.757574 2560 generic.go:356] "Generic (PLEG): container finished" podID="346bb733-1a4a-4359-bbdd-86af8235f1bc" containerID="6373e0b836b5f42ed27f57ff3bfc313f746ed039af3c1fc94c58bafe45c3f1f4" exitCode=0 Jul 24 00:48:42.757839 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.757624 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" Jul 24 00:48:42.757839 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.757672 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" event={"ID":"346bb733-1a4a-4359-bbdd-86af8235f1bc","Type":"ContainerDied","Data":"6373e0b836b5f42ed27f57ff3bfc313f746ed039af3c1fc94c58bafe45c3f1f4"} Jul 24 00:48:42.757839 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.757737 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5" event={"ID":"346bb733-1a4a-4359-bbdd-86af8235f1bc","Type":"ContainerDied","Data":"8467803f528e1e3635761a405646a9146c26bd145de1c1ba234bd1242e1d2b70"} Jul 24 00:48:42.757839 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.757771 2560 scope.go:117] "RemoveContainer" containerID="6373e0b836b5f42ed27f57ff3bfc313f746ed039af3c1fc94c58bafe45c3f1f4" Jul 24 00:48:42.758963 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.758944 2560 generic.go:356] "Generic (PLEG): container finished" podID="a7421728-99ca-4881-8c76-755fcad216fb" containerID="01b9ba4f73d64e2f3c5e9a274ed66c8b18dd492d56551f6ab48e63fe588cde96" exitCode=0 Jul 24 00:48:42.759043 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.759009 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" Jul 24 00:48:42.759043 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.759022 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" event={"ID":"a7421728-99ca-4881-8c76-755fcad216fb","Type":"ContainerDied","Data":"01b9ba4f73d64e2f3c5e9a274ed66c8b18dd492d56551f6ab48e63fe588cde96"} Jul 24 00:48:42.759149 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.759053 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk" event={"ID":"a7421728-99ca-4881-8c76-755fcad216fb","Type":"ContainerDied","Data":"364aaa80c91efd37175ba828aff679fef6745aa9c7255cd3b8fe7ba64aaf0245"} Jul 24 00:48:42.766878 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.766856 2560 scope.go:117] "RemoveContainer" containerID="6373e0b836b5f42ed27f57ff3bfc313f746ed039af3c1fc94c58bafe45c3f1f4" Jul 24 00:48:42.767203 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:48:42.767177 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6373e0b836b5f42ed27f57ff3bfc313f746ed039af3c1fc94c58bafe45c3f1f4\": container with ID starting with 6373e0b836b5f42ed27f57ff3bfc313f746ed039af3c1fc94c58bafe45c3f1f4 not found: ID does not exist" containerID="6373e0b836b5f42ed27f57ff3bfc313f746ed039af3c1fc94c58bafe45c3f1f4" Jul 24 00:48:42.767326 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.767211 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6373e0b836b5f42ed27f57ff3bfc313f746ed039af3c1fc94c58bafe45c3f1f4"} err="failed to get container status \"6373e0b836b5f42ed27f57ff3bfc313f746ed039af3c1fc94c58bafe45c3f1f4\": rpc error: code = NotFound desc = could not find container \"6373e0b836b5f42ed27f57ff3bfc313f746ed039af3c1fc94c58bafe45c3f1f4\": container with ID starting with 6373e0b836b5f42ed27f57ff3bfc313f746ed039af3c1fc94c58bafe45c3f1f4 not found: ID does not exist" Jul 24 00:48:42.767326 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.767234 2560 scope.go:117] "RemoveContainer" containerID="01b9ba4f73d64e2f3c5e9a274ed66c8b18dd492d56551f6ab48e63fe588cde96" Jul 24 00:48:42.775877 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.775856 2560 scope.go:117] "RemoveContainer" containerID="01b9ba4f73d64e2f3c5e9a274ed66c8b18dd492d56551f6ab48e63fe588cde96" Jul 24 00:48:42.776141 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:48:42.776123 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01b9ba4f73d64e2f3c5e9a274ed66c8b18dd492d56551f6ab48e63fe588cde96\": container with ID starting with 01b9ba4f73d64e2f3c5e9a274ed66c8b18dd492d56551f6ab48e63fe588cde96 not found: ID does not exist" containerID="01b9ba4f73d64e2f3c5e9a274ed66c8b18dd492d56551f6ab48e63fe588cde96" Jul 24 00:48:42.776207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.776147 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01b9ba4f73d64e2f3c5e9a274ed66c8b18dd492d56551f6ab48e63fe588cde96"} err="failed to get container status \"01b9ba4f73d64e2f3c5e9a274ed66c8b18dd492d56551f6ab48e63fe588cde96\": rpc error: code = NotFound desc = could not find container \"01b9ba4f73d64e2f3c5e9a274ed66c8b18dd492d56551f6ab48e63fe588cde96\": container with ID starting with 01b9ba4f73d64e2f3c5e9a274ed66c8b18dd492d56551f6ab48e63fe588cde96 not found: ID does not exist" Jul 24 00:48:42.784086 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.784063 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5"] Jul 24 00:48:42.788411 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.788389 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5f040e-56d4cdf45b-d97x5"] Jul 24 00:48:42.799364 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.799341 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk"] Jul 24 00:48:42.801417 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:42.801394 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5f040e-7674674676-f6kwk"] Jul 24 00:48:44.348213 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:44.348183 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="346bb733-1a4a-4359-bbdd-86af8235f1bc" path="/var/lib/kubelet/pods/346bb733-1a4a-4359-bbdd-86af8235f1bc/volumes" Jul 24 00:48:44.348607 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:44.348553 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7421728-99ca-4881-8c76-755fcad216fb" path="/var/lib/kubelet/pods/a7421728-99ca-4881-8c76-755fcad216fb/volumes" Jul 24 00:48:46.543033 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:46.542999 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:46.957123 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:46.957021 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:52.686529 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:52.686497 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:52.748198 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:52.748158 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:52.965538 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:52.965431 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm"] Jul 24 00:48:52.965753 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:52.965722 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" podUID="94d34c32-cf1b-4de9-b07f-5ba888b14f00" containerName="istio-proxy" containerID="cri-o://4df08d9f2161e45ff555cb322dbf4f942bf88dece2b09f813e710a0b7253d52d" gracePeriod=30 Jul 24 00:48:52.990322 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:52.990293 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:53.248527 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:53.248497 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:53.635917 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:53.635831 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:53.990076 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:53.989995 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:54.945221 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:54.945185 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:55.592087 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:55.592041 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:56.838588 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:56.838554 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:56.926799 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:56.926766 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:56.995750 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:56.995713 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:57.486992 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.486952 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:57.668736 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.668698 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:57.762390 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.762353 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:48:57.763508 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.763487 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq"] Jul 24 00:48:57.763905 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.763888 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b84739b4-1efc-4f75-beff-3a256fa45df4" containerName="payload-processing" Jul 24 00:48:57.763905 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.763902 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b84739b4-1efc-4f75-beff-3a256fa45df4" containerName="payload-processing" Jul 24 00:48:57.764071 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.763913 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7421728-99ca-4881-8c76-755fcad216fb" containerName="payload-pre-processing" Jul 24 00:48:57.764071 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.763919 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7421728-99ca-4881-8c76-755fcad216fb" containerName="payload-pre-processing" Jul 24 00:48:57.764071 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.763930 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cdd7205-507d-4daa-b081-e785472fb1b8" containerName="payload-pre-processing" Jul 24 00:48:57.764071 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.763936 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cdd7205-507d-4daa-b081-e785472fb1b8" containerName="payload-pre-processing" Jul 24 00:48:57.764071 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.763947 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="346bb733-1a4a-4359-bbdd-86af8235f1bc" containerName="payload-processing" Jul 24 00:48:57.764071 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.763952 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="346bb733-1a4a-4359-bbdd-86af8235f1bc" containerName="payload-processing" Jul 24 00:48:57.764071 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.764019 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7421728-99ca-4881-8c76-755fcad216fb" containerName="payload-pre-processing" Jul 24 00:48:57.764071 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.764027 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cdd7205-507d-4daa-b081-e785472fb1b8" containerName="payload-pre-processing" Jul 24 00:48:57.764071 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.764034 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b84739b4-1efc-4f75-beff-3a256fa45df4" containerName="payload-processing" Jul 24 00:48:57.764071 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.764044 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="346bb733-1a4a-4359-bbdd-86af8235f1bc" containerName="payload-processing" Jul 24 00:48:57.766897 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.766883 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" Jul 24 00:48:57.769332 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.769310 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-796b6a-dockercfg-pv5fh\"" Jul 24 00:48:57.769439 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.769322 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-796b6a\"" Jul 24 00:48:57.777184 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.777154 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq"] Jul 24 00:48:57.801338 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.801301 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx"] Jul 24 00:48:57.805340 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.805319 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" Jul 24 00:48:57.820132 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.820101 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx"] Jul 24 00:48:57.921925 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.921891 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jvpm\" (UniqueName: \"kubernetes.io/projected/3b6f4083-2430-446d-9960-3c05e7cbdb7e-kube-api-access-2jvpm\") pod \"payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx\" (UID: \"3b6f4083-2430-446d-9960-3c05e7cbdb7e\") " pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" Jul 24 00:48:57.922301 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.922021 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b6f4083-2430-446d-9960-3c05e7cbdb7e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx\" (UID: \"3b6f4083-2430-446d-9960-3c05e7cbdb7e\") " pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" Jul 24 00:48:57.922301 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.922058 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7dv5\" (UniqueName: \"kubernetes.io/projected/d8558844-6931-4eac-a660-380c1cd94130-kube-api-access-s7dv5\") pod \"payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq\" (UID: \"d8558844-6931-4eac-a660-380c1cd94130\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" Jul 24 00:48:57.922301 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:57.922139 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8558844-6931-4eac-a660-380c1cd94130-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq\" (UID: \"d8558844-6931-4eac-a660-380c1cd94130\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" Jul 24 00:48:58.022615 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.022543 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b6f4083-2430-446d-9960-3c05e7cbdb7e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx\" (UID: \"3b6f4083-2430-446d-9960-3c05e7cbdb7e\") " pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" Jul 24 00:48:58.022615 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.022584 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7dv5\" (UniqueName: \"kubernetes.io/projected/d8558844-6931-4eac-a660-380c1cd94130-kube-api-access-s7dv5\") pod \"payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq\" (UID: \"d8558844-6931-4eac-a660-380c1cd94130\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" Jul 24 00:48:58.022821 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.022629 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8558844-6931-4eac-a660-380c1cd94130-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq\" (UID: \"d8558844-6931-4eac-a660-380c1cd94130\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" Jul 24 00:48:58.022821 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.022681 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jvpm\" (UniqueName: \"kubernetes.io/projected/3b6f4083-2430-446d-9960-3c05e7cbdb7e-kube-api-access-2jvpm\") pod \"payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx\" (UID: \"3b6f4083-2430-446d-9960-3c05e7cbdb7e\") " pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" Jul 24 00:48:58.023313 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.023289 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b6f4083-2430-446d-9960-3c05e7cbdb7e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx\" (UID: \"3b6f4083-2430-446d-9960-3c05e7cbdb7e\") " pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" Jul 24 00:48:58.023445 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.023423 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8558844-6931-4eac-a660-380c1cd94130-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq\" (UID: \"d8558844-6931-4eac-a660-380c1cd94130\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" Jul 24 00:48:58.030148 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.030121 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7dv5\" (UniqueName: \"kubernetes.io/projected/d8558844-6931-4eac-a660-380c1cd94130-kube-api-access-s7dv5\") pod \"payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq\" (UID: \"d8558844-6931-4eac-a660-380c1cd94130\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" Jul 24 00:48:58.030402 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.030380 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jvpm\" (UniqueName: \"kubernetes.io/projected/3b6f4083-2430-446d-9960-3c05e7cbdb7e-kube-api-access-2jvpm\") pod \"payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx\" (UID: \"3b6f4083-2430-446d-9960-3c05e7cbdb7e\") " pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" Jul 24 00:48:58.078795 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.078761 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" Jul 24 00:48:58.116048 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.115678 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" Jul 24 00:48:58.236589 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.236538 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq"] Jul 24 00:48:58.241515 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:48:58.241479 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8558844_6931_4eac_a660_380c1cd94130.slice/crio-ee2a00dac6df867aa450f1144e23ebdea77a227ba89a1ef3f0881d739bf703eb WatchSource:0}: Error finding container ee2a00dac6df867aa450f1144e23ebdea77a227ba89a1ef3f0881d739bf703eb: Status 404 returned error can't find the container with id ee2a00dac6df867aa450f1144e23ebdea77a227ba89a1ef3f0881d739bf703eb Jul 24 00:48:58.253051 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.253029 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:48:58.280927 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.280903 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx"] Jul 24 00:48:58.283343 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:48:58.283311 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b6f4083_2430_446d_9960_3c05e7cbdb7e.slice/crio-ecf6f6210cb5e788235563ab367fa431378e9b6c377c386f9a7ea1ae0853fb03 WatchSource:0}: Error finding container ecf6f6210cb5e788235563ab367fa431378e9b6c377c386f9a7ea1ae0853fb03: Status 404 returned error can't find the container with id ecf6f6210cb5e788235563ab367fa431378e9b6c377c386f9a7ea1ae0853fb03 Jul 24 00:48:58.325635 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.325609 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-envoy\") pod \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " Jul 24 00:48:58.325802 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.325660 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-podinfo\") pod \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " Jul 24 00:48:58.325802 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.325691 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-credential-socket\") pod \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " Jul 24 00:48:58.325802 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.325741 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istiod-ca-cert\") pod \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " Jul 24 00:48:58.325802 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.325799 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-data\") pod \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " Jul 24 00:48:58.326038 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.325825 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8zm7\" (UniqueName: \"kubernetes.io/projected/94d34c32-cf1b-4de9-b07f-5ba888b14f00-kube-api-access-s8zm7\") pod \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " Jul 24 00:48:58.326038 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.325861 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-token\") pod \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " Jul 24 00:48:58.326038 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.325882 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-workload-certs\") pod \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " Jul 24 00:48:58.326038 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.325902 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-workload-socket\") pod \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\" (UID: \"94d34c32-cf1b-4de9-b07f-5ba888b14f00\") " Jul 24 00:48:58.326256 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.326103 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "94d34c32-cf1b-4de9-b07f-5ba888b14f00" (UID: "94d34c32-cf1b-4de9-b07f-5ba888b14f00"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:48:58.326327 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.326273 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-credential-socket\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:58.326327 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.326303 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "94d34c32-cf1b-4de9-b07f-5ba888b14f00" (UID: "94d34c32-cf1b-4de9-b07f-5ba888b14f00"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:48:58.326479 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.326457 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "94d34c32-cf1b-4de9-b07f-5ba888b14f00" (UID: "94d34c32-cf1b-4de9-b07f-5ba888b14f00"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:48:58.326540 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.326474 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "94d34c32-cf1b-4de9-b07f-5ba888b14f00" (UID: "94d34c32-cf1b-4de9-b07f-5ba888b14f00"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:48:58.326879 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.326852 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-data" (OuterVolumeSpecName: "istio-data") pod "94d34c32-cf1b-4de9-b07f-5ba888b14f00" (UID: "94d34c32-cf1b-4de9-b07f-5ba888b14f00"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:48:58.328235 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.328208 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94d34c32-cf1b-4de9-b07f-5ba888b14f00-kube-api-access-s8zm7" (OuterVolumeSpecName: "kube-api-access-s8zm7") pod "94d34c32-cf1b-4de9-b07f-5ba888b14f00" (UID: "94d34c32-cf1b-4de9-b07f-5ba888b14f00"). InnerVolumeSpecName "kube-api-access-s8zm7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:48:58.328322 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.328277 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-token" (OuterVolumeSpecName: "istio-token") pod "94d34c32-cf1b-4de9-b07f-5ba888b14f00" (UID: "94d34c32-cf1b-4de9-b07f-5ba888b14f00"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:48:58.328322 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.328298 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "94d34c32-cf1b-4de9-b07f-5ba888b14f00" (UID: "94d34c32-cf1b-4de9-b07f-5ba888b14f00"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:48:58.328396 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.328302 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "94d34c32-cf1b-4de9-b07f-5ba888b14f00" (UID: "94d34c32-cf1b-4de9-b07f-5ba888b14f00"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 00:48:58.427537 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.427502 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-data\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:58.427537 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.427534 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8zm7\" (UniqueName: \"kubernetes.io/projected/94d34c32-cf1b-4de9-b07f-5ba888b14f00-kube-api-access-s8zm7\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:58.427537 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.427543 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-token\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:58.427791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.427552 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-workload-certs\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:58.427791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.427560 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-workload-socket\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:58.427791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.427568 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-envoy\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:58.427791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.427578 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istio-podinfo\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:58.427791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.427585 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94d34c32-cf1b-4de9-b07f-5ba888b14f00-istiod-ca-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:48:58.849547 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.849510 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" event={"ID":"3b6f4083-2430-446d-9960-3c05e7cbdb7e","Type":"ContainerStarted","Data":"952c3c7c8a125f75cc4b1f93f01f3e555ea096f98064c5ca96349f87d460f3de"} Jul 24 00:48:58.849777 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.849561 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" event={"ID":"3b6f4083-2430-446d-9960-3c05e7cbdb7e","Type":"ContainerStarted","Data":"ecf6f6210cb5e788235563ab367fa431378e9b6c377c386f9a7ea1ae0853fb03"} Jul 24 00:48:58.849777 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.849715 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" Jul 24 00:48:58.851180 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.851155 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" event={"ID":"d8558844-6931-4eac-a660-380c1cd94130","Type":"ContainerStarted","Data":"7976a6e9d5aee9677c9ff8e3d8da76784f27bfe21469cc0a986ebd416ccfd4c3"} Jul 24 00:48:58.851305 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.851202 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" Jul 24 00:48:58.851305 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.851223 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" event={"ID":"d8558844-6931-4eac-a660-380c1cd94130","Type":"ContainerStarted","Data":"ee2a00dac6df867aa450f1144e23ebdea77a227ba89a1ef3f0881d739bf703eb"} Jul 24 00:48:58.852693 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.852672 2560 generic.go:356] "Generic (PLEG): container finished" podID="94d34c32-cf1b-4de9-b07f-5ba888b14f00" containerID="4df08d9f2161e45ff555cb322dbf4f942bf88dece2b09f813e710a0b7253d52d" exitCode=0 Jul 24 00:48:58.852823 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.852724 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" event={"ID":"94d34c32-cf1b-4de9-b07f-5ba888b14f00","Type":"ContainerDied","Data":"4df08d9f2161e45ff555cb322dbf4f942bf88dece2b09f813e710a0b7253d52d"} Jul 24 00:48:58.852823 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.852733 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" Jul 24 00:48:58.852823 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.852753 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm" event={"ID":"94d34c32-cf1b-4de9-b07f-5ba888b14f00","Type":"ContainerDied","Data":"bff59756af62cfa5a3fa68dff2c0de7d21264a74f83fb4bc25c6c883efa527fa"} Jul 24 00:48:58.852823 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.852775 2560 scope.go:117] "RemoveContainer" containerID="4df08d9f2161e45ff555cb322dbf4f942bf88dece2b09f813e710a0b7253d52d" Jul 24 00:48:58.864661 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.864627 2560 scope.go:117] "RemoveContainer" containerID="1a2602a72a1b09932f040e9d6d0c39c3d1bfbdb393262c0ad9bbf49eccb49504" Jul 24 00:48:58.867959 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.867914 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" podStartSLOduration=1.867900762 podStartE2EDuration="1.867900762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:48:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:48:58.866473277 +0000 UTC m=+2267.471787595" watchObservedRunningTime="2026-07-24 00:48:58.867900762 +0000 UTC m=+2267.473215078" Jul 24 00:48:58.874120 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.874097 2560 scope.go:117] "RemoveContainer" containerID="4df08d9f2161e45ff555cb322dbf4f942bf88dece2b09f813e710a0b7253d52d" Jul 24 00:48:58.874442 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:48:58.874423 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4df08d9f2161e45ff555cb322dbf4f942bf88dece2b09f813e710a0b7253d52d\": container with ID starting with 4df08d9f2161e45ff555cb322dbf4f942bf88dece2b09f813e710a0b7253d52d not found: ID does not exist" containerID="4df08d9f2161e45ff555cb322dbf4f942bf88dece2b09f813e710a0b7253d52d" Jul 24 00:48:58.874517 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.874450 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4df08d9f2161e45ff555cb322dbf4f942bf88dece2b09f813e710a0b7253d52d"} err="failed to get container status \"4df08d9f2161e45ff555cb322dbf4f942bf88dece2b09f813e710a0b7253d52d\": rpc error: code = NotFound desc = could not find container \"4df08d9f2161e45ff555cb322dbf4f942bf88dece2b09f813e710a0b7253d52d\": container with ID starting with 4df08d9f2161e45ff555cb322dbf4f942bf88dece2b09f813e710a0b7253d52d not found: ID does not exist" Jul 24 00:48:58.874517 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.874468 2560 scope.go:117] "RemoveContainer" containerID="1a2602a72a1b09932f040e9d6d0c39c3d1bfbdb393262c0ad9bbf49eccb49504" Jul 24 00:48:58.874792 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:48:58.874775 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a2602a72a1b09932f040e9d6d0c39c3d1bfbdb393262c0ad9bbf49eccb49504\": container with ID starting with 1a2602a72a1b09932f040e9d6d0c39c3d1bfbdb393262c0ad9bbf49eccb49504 not found: ID does not exist" containerID="1a2602a72a1b09932f040e9d6d0c39c3d1bfbdb393262c0ad9bbf49eccb49504" Jul 24 00:48:58.874850 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.874797 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a2602a72a1b09932f040e9d6d0c39c3d1bfbdb393262c0ad9bbf49eccb49504"} err="failed to get container status \"1a2602a72a1b09932f040e9d6d0c39c3d1bfbdb393262c0ad9bbf49eccb49504\": rpc error: code = NotFound desc = could not find container \"1a2602a72a1b09932f040e9d6d0c39c3d1bfbdb393262c0ad9bbf49eccb49504\": container with ID starting with 1a2602a72a1b09932f040e9d6d0c39c3d1bfbdb393262c0ad9bbf49eccb49504 not found: ID does not exist" Jul 24 00:48:58.881826 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.881776 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" podStartSLOduration=1.881759189 podStartE2EDuration="1.881759189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:48:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:48:58.8811576 +0000 UTC m=+2267.486471919" watchObservedRunningTime="2026-07-24 00:48:58.881759189 +0000 UTC m=+2267.487073506" Jul 24 00:48:58.894783 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.894726 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm"] Jul 24 00:48:58.900248 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:48:58.900222 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-5f040e-openshift-default-5974bb9864-dtbmm"] Jul 24 00:49:00.348515 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:00.348473 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94d34c32-cf1b-4de9-b07f-5ba888b14f00" path="/var/lib/kubelet/pods/94d34c32-cf1b-4de9-b07f-5ba888b14f00/volumes" Jul 24 00:49:01.391702 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:01.391673 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:01.923812 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:01.923780 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:02.313346 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.313314 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb"] Jul 24 00:49:02.313749 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.313735 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94d34c32-cf1b-4de9-b07f-5ba888b14f00" containerName="istio-proxy" Jul 24 00:49:02.313793 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.313752 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="94d34c32-cf1b-4de9-b07f-5ba888b14f00" containerName="istio-proxy" Jul 24 00:49:02.313793 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.313771 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94d34c32-cf1b-4de9-b07f-5ba888b14f00" containerName="istio-proxy" Jul 24 00:49:02.313793 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.313777 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="94d34c32-cf1b-4de9-b07f-5ba888b14f00" containerName="istio-proxy" Jul 24 00:49:02.313896 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.313832 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="94d34c32-cf1b-4de9-b07f-5ba888b14f00" containerName="istio-proxy" Jul 24 00:49:02.318421 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.318398 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.324104 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.324079 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a4c256-openshift-default-dockercfg-qm4sd\"" Jul 24 00:49:02.338670 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.338627 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb"] Jul 24 00:49:02.362078 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.362043 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7f81045-8cde-4571-b37e-356951980b2f-istio-token\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.362294 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.362114 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-workload-socket\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.362294 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.362184 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7f81045-8cde-4571-b37e-356951980b2f-istiod-ca-cert\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.362527 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.362407 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-istio-data\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.362721 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.362674 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-istio-envoy\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.362912 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.362781 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-workload-certs\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.363011 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.362909 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qm47\" (UniqueName: \"kubernetes.io/projected/d7f81045-8cde-4571-b37e-356951980b2f-kube-api-access-7qm47\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.363518 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.363493 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-credential-socket\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.363629 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.363541 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7f81045-8cde-4571-b37e-356951980b2f-istio-podinfo\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.432371 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.432336 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:02.464543 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.464505 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-istio-data\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.464543 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.464548 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-istio-envoy\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.464809 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.464587 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-workload-certs\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.464809 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.464626 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qm47\" (UniqueName: \"kubernetes.io/projected/d7f81045-8cde-4571-b37e-356951980b2f-kube-api-access-7qm47\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.464809 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.464677 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-credential-socket\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.464809 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.464707 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7f81045-8cde-4571-b37e-356951980b2f-istio-podinfo\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.464809 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.464773 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7f81045-8cde-4571-b37e-356951980b2f-istio-token\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.464809 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.464807 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-workload-socket\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.465139 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.464845 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7f81045-8cde-4571-b37e-356951980b2f-istiod-ca-cert\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.465139 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.464978 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-istio-data\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.465139 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.465056 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-workload-certs\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.465374 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.465351 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-credential-socket\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.465434 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.465397 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-workload-socket\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.465547 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.465529 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7f81045-8cde-4571-b37e-356951980b2f-istiod-ca-cert\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.467087 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.467069 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-istio-envoy\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.467225 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.467205 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7f81045-8cde-4571-b37e-356951980b2f-istio-podinfo\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.472884 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.472851 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7f81045-8cde-4571-b37e-356951980b2f-istio-token\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.473003 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.472857 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qm47\" (UniqueName: \"kubernetes.io/projected/d7f81045-8cde-4571-b37e-356951980b2f-kube-api-access-7qm47\") pod \"e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.628526 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.628436 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:02.770832 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.770798 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb"] Jul 24 00:49:02.773079 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:49:02.773051 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7f81045_8cde_4571_b37e_356951980b2f.slice/crio-8ebc2e53ccb1493eb71667d7c94fc987a7f041bacceee456185000782cec205b WatchSource:0}: Error finding container 8ebc2e53ccb1493eb71667d7c94fc987a7f041bacceee456185000782cec205b: Status 404 returned error can't find the container with id 8ebc2e53ccb1493eb71667d7c94fc987a7f041bacceee456185000782cec205b Jul 24 00:49:02.775847 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.775814 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:49:02.775947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.775877 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:49:02.775947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.775909 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:49:02.877937 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.877902 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" event={"ID":"d7f81045-8cde-4571-b37e-356951980b2f","Type":"ContainerStarted","Data":"e3b75badc6bf665a64f52e77c9d2b71449bcca28cd17d9021d3ae0f1c74f6a5e"} Jul 24 00:49:02.878120 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.877960 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" event={"ID":"d7f81045-8cde-4571-b37e-356951980b2f","Type":"ContainerStarted","Data":"8ebc2e53ccb1493eb71667d7c94fc987a7f041bacceee456185000782cec205b"} Jul 24 00:49:02.911740 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.911619 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" podStartSLOduration=0.91160105 podStartE2EDuration="911.60105ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:49:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:49:02.910689108 +0000 UTC m=+2271.516003425" watchObservedRunningTime="2026-07-24 00:49:02.91160105 +0000 UTC m=+2271.516915365" Jul 24 00:49:02.972212 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:02.972176 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:03.629254 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:03.629210 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:03.634214 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:03.634182 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:03.883740 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:03.883626 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:03.884609 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:03.884592 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:05.004137 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.004095 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:05.093971 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.093928 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:05.169150 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.169111 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:05.586448 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.586414 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:05.790619 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.790586 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:05.907794 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.907714 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr"] Jul 24 00:49:05.908395 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.908368 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="94d34c32-cf1b-4de9-b07f-5ba888b14f00" containerName="istio-proxy" Jul 24 00:49:05.911467 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.911445 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:05.911626 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.911609 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" Jul 24 00:49:05.917489 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.917465 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-a4c256-dockercfg-b976f\"" Jul 24 00:49:05.917843 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.917825 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-a4c256\"" Jul 24 00:49:05.928540 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.928516 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr"] Jul 24 00:49:05.933784 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.933758 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm"] Jul 24 00:49:05.942890 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.942857 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" Jul 24 00:49:05.954078 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.954042 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm"] Jul 24 00:49:05.998868 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.998833 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8ngs\" (UniqueName: \"kubernetes.io/projected/9c652f42-aaca-4209-a6f6-f53386b8292f-kube-api-access-b8ngs\") pod \"payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm\" (UID: \"9c652f42-aaca-4209-a6f6-f53386b8292f\") " pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" Jul 24 00:49:05.999074 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.999039 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daf1fb43-520d-4bad-b03a-3cb8c36f5b39-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr\" (UID: \"daf1fb43-520d-4bad-b03a-3cb8c36f5b39\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" Jul 24 00:49:05.999143 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.999103 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c652f42-aaca-4209-a6f6-f53386b8292f-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm\" (UID: \"9c652f42-aaca-4209-a6f6-f53386b8292f\") " pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" Jul 24 00:49:05.999230 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:05.999202 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxjtj\" (UniqueName: \"kubernetes.io/projected/daf1fb43-520d-4bad-b03a-3cb8c36f5b39-kube-api-access-wxjtj\") pod \"payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr\" (UID: \"daf1fb43-520d-4bad-b03a-3cb8c36f5b39\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" Jul 24 00:49:06.100398 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.100361 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8ngs\" (UniqueName: \"kubernetes.io/projected/9c652f42-aaca-4209-a6f6-f53386b8292f-kube-api-access-b8ngs\") pod \"payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm\" (UID: \"9c652f42-aaca-4209-a6f6-f53386b8292f\") " pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" Jul 24 00:49:06.100861 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.100451 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daf1fb43-520d-4bad-b03a-3cb8c36f5b39-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr\" (UID: \"daf1fb43-520d-4bad-b03a-3cb8c36f5b39\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" Jul 24 00:49:06.100861 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.100476 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c652f42-aaca-4209-a6f6-f53386b8292f-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm\" (UID: \"9c652f42-aaca-4209-a6f6-f53386b8292f\") " pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" Jul 24 00:49:06.100861 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.100509 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxjtj\" (UniqueName: \"kubernetes.io/projected/daf1fb43-520d-4bad-b03a-3cb8c36f5b39-kube-api-access-wxjtj\") pod \"payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr\" (UID: \"daf1fb43-520d-4bad-b03a-3cb8c36f5b39\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" Jul 24 00:49:06.101243 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.101218 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daf1fb43-520d-4bad-b03a-3cb8c36f5b39-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr\" (UID: \"daf1fb43-520d-4bad-b03a-3cb8c36f5b39\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" Jul 24 00:49:06.101332 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.101274 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c652f42-aaca-4209-a6f6-f53386b8292f-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm\" (UID: \"9c652f42-aaca-4209-a6f6-f53386b8292f\") " pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" Jul 24 00:49:06.108973 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.108948 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8ngs\" (UniqueName: \"kubernetes.io/projected/9c652f42-aaca-4209-a6f6-f53386b8292f-kube-api-access-b8ngs\") pod \"payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm\" (UID: \"9c652f42-aaca-4209-a6f6-f53386b8292f\") " pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" Jul 24 00:49:06.109086 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.109007 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxjtj\" (UniqueName: \"kubernetes.io/projected/daf1fb43-520d-4bad-b03a-3cb8c36f5b39-kube-api-access-wxjtj\") pod \"payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr\" (UID: \"daf1fb43-520d-4bad-b03a-3cb8c36f5b39\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" Jul 24 00:49:06.222307 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.222198 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" Jul 24 00:49:06.256737 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.256693 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" Jul 24 00:49:06.392456 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.392183 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr"] Jul 24 00:49:06.395833 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:49:06.395778 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaf1fb43_520d_4bad_b03a_3cb8c36f5b39.slice/crio-c14c9c3fdb4410c3619f97aaf722c261884ec9a08133cc591ff1d37425982e6e WatchSource:0}: Error finding container c14c9c3fdb4410c3619f97aaf722c261884ec9a08133cc591ff1d37425982e6e: Status 404 returned error can't find the container with id c14c9c3fdb4410c3619f97aaf722c261884ec9a08133cc591ff1d37425982e6e Jul 24 00:49:06.422819 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.422795 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm"] Jul 24 00:49:06.431864 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:49:06.431837 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c652f42_aaca_4209_a6f6_f53386b8292f.slice/crio-078a2cef546eb1ca9fe7d9b213ca49f793f2d58e0f045e94e584b06ee6741fa6 WatchSource:0}: Error finding container 078a2cef546eb1ca9fe7d9b213ca49f793f2d58e0f045e94e584b06ee6741fa6: Status 404 returned error can't find the container with id 078a2cef546eb1ca9fe7d9b213ca49f793f2d58e0f045e94e584b06ee6741fa6 Jul 24 00:49:06.901332 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.901290 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" event={"ID":"9c652f42-aaca-4209-a6f6-f53386b8292f","Type":"ContainerStarted","Data":"ef14d6abce1b311439af3d3cf07f39230629674c309d8f2520956ec03ae6362e"} Jul 24 00:49:06.901542 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.901343 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" event={"ID":"9c652f42-aaca-4209-a6f6-f53386b8292f","Type":"ContainerStarted","Data":"078a2cef546eb1ca9fe7d9b213ca49f793f2d58e0f045e94e584b06ee6741fa6"} Jul 24 00:49:06.901542 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.901486 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" Jul 24 00:49:06.902987 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.902961 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" event={"ID":"daf1fb43-520d-4bad-b03a-3cb8c36f5b39","Type":"ContainerStarted","Data":"f351e05273f6823969ebdd47c2cfe493b9788cc88e7a70f27d109eac610745a0"} Jul 24 00:49:06.903108 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.903004 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" event={"ID":"daf1fb43-520d-4bad-b03a-3cb8c36f5b39","Type":"ContainerStarted","Data":"c14c9c3fdb4410c3619f97aaf722c261884ec9a08133cc591ff1d37425982e6e"} Jul 24 00:49:06.903108 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.903041 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" Jul 24 00:49:06.927253 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.927202 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" podStartSLOduration=1.927186018 podStartE2EDuration="1.927186018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:49:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:49:06.925611936 +0000 UTC m=+2275.530926252" watchObservedRunningTime="2026-07-24 00:49:06.927186018 +0000 UTC m=+2275.532500334" Jul 24 00:49:06.947396 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:06.947342 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" podStartSLOduration=1.9473216500000001 podStartE2EDuration="1.94732165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:49:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:49:06.945886927 +0000 UTC m=+2275.551201245" watchObservedRunningTime="2026-07-24 00:49:06.94732165 +0000 UTC m=+2275.552635966" Jul 24 00:49:09.469483 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:09.469444 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:09.860879 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:09.860848 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" Jul 24 00:49:09.861086 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:09.861001 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" Jul 24 00:49:13.805582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:13.805545 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:13.864982 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:13.864948 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:17.911776 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:17.911732 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" Jul 24 00:49:17.912167 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:17.911805 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" Jul 24 00:49:19.925874 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:19.925827 2560 patch_prober.go:28] interesting pod/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 00:49:19.926306 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:19.925899 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" podUID="d7f81045-8cde-4571-b37e-356951980b2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 00:49:21.986944 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:21.986910 2560 generic.go:356] "Generic (PLEG): container finished" podID="d7f81045-8cde-4571-b37e-356951980b2f" containerID="e3b75badc6bf665a64f52e77c9d2b71449bcca28cd17d9021d3ae0f1c74f6a5e" exitCode=0 Jul 24 00:49:21.987453 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:21.986955 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" event={"ID":"d7f81045-8cde-4571-b37e-356951980b2f","Type":"ContainerDied","Data":"e3b75badc6bf665a64f52e77c9d2b71449bcca28cd17d9021d3ae0f1c74f6a5e"} Jul 24 00:49:21.987453 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:21.987354 2560 scope.go:117] "RemoveContainer" containerID="e3b75badc6bf665a64f52e77c9d2b71449bcca28cd17d9021d3ae0f1c74f6a5e" Jul 24 00:49:21.988404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:21.988373 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:49:21.988455 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:21.988444 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:49:21.988495 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:21.988474 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:49:22.997154 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:22.997116 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" event={"ID":"d7f81045-8cde-4571-b37e-356951980b2f","Type":"ContainerStarted","Data":"f7a0ecec002c42a3dddd3173e80cb93c189cfe9bf5f5144eebd2a713958e2719"} Jul 24 00:49:23.629291 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:23.629242 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:23.629546 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:23.629463 2560 patch_prober.go:28] interesting pod/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 24 00:49:23.629546 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:23.629512 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" podUID="d7f81045-8cde-4571-b37e-356951980b2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 24 00:49:24.629841 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:24.629799 2560 patch_prober.go:28] interesting pod/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 24 00:49:24.630263 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:24.629859 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" podUID="d7f81045-8cde-4571-b37e-356951980b2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 24 00:49:25.629236 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:25.629196 2560 patch_prober.go:28] interesting pod/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 24 00:49:25.629436 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:25.629255 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" podUID="d7f81045-8cde-4571-b37e-356951980b2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 24 00:49:26.629077 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:26.629036 2560 patch_prober.go:28] interesting pod/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 24 00:49:26.629574 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:26.629102 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" podUID="d7f81045-8cde-4571-b37e-356951980b2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 24 00:49:27.632910 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:27.632874 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:27.633300 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:27.633203 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:27.634014 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:27.633995 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:49:33.433623 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.433585 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:33.456844 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.456802 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx"] Jul 24 00:49:33.458932 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.458896 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" podUID="3b6f4083-2430-446d-9960-3c05e7cbdb7e" containerName="payload-processing" containerID="cri-o://952c3c7c8a125f75cc4b1f93f01f3e555ea096f98064c5ca96349f87d460f3de" gracePeriod=30 Jul 24 00:49:33.461678 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.461253 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq"] Jul 24 00:49:33.461678 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.461471 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" podUID="d8558844-6931-4eac-a660-380c1cd94130" containerName="payload-pre-processing" containerID="cri-o://7976a6e9d5aee9677c9ff8e3d8da76784f27bfe21469cc0a986ebd416ccfd4c3" gracePeriod=30 Jul 24 00:49:33.504911 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.504881 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:33.566437 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.566403 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:33.741292 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.741265 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" Jul 24 00:49:33.768880 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.768851 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" Jul 24 00:49:33.893978 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.893943 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8558844-6931-4eac-a660-380c1cd94130-plugins-config-volume\") pod \"d8558844-6931-4eac-a660-380c1cd94130\" (UID: \"d8558844-6931-4eac-a660-380c1cd94130\") " Jul 24 00:49:33.894163 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.894006 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7dv5\" (UniqueName: \"kubernetes.io/projected/d8558844-6931-4eac-a660-380c1cd94130-kube-api-access-s7dv5\") pod \"d8558844-6931-4eac-a660-380c1cd94130\" (UID: \"d8558844-6931-4eac-a660-380c1cd94130\") " Jul 24 00:49:33.894163 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.894045 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b6f4083-2430-446d-9960-3c05e7cbdb7e-plugins-config-volume\") pod \"3b6f4083-2430-446d-9960-3c05e7cbdb7e\" (UID: \"3b6f4083-2430-446d-9960-3c05e7cbdb7e\") " Jul 24 00:49:33.894163 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.894132 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jvpm\" (UniqueName: \"kubernetes.io/projected/3b6f4083-2430-446d-9960-3c05e7cbdb7e-kube-api-access-2jvpm\") pod \"3b6f4083-2430-446d-9960-3c05e7cbdb7e\" (UID: \"3b6f4083-2430-446d-9960-3c05e7cbdb7e\") " Jul 24 00:49:33.894432 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.894399 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8558844-6931-4eac-a660-380c1cd94130-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d8558844-6931-4eac-a660-380c1cd94130" (UID: "d8558844-6931-4eac-a660-380c1cd94130"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:49:33.894558 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.894533 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b6f4083-2430-446d-9960-3c05e7cbdb7e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3b6f4083-2430-446d-9960-3c05e7cbdb7e" (UID: "3b6f4083-2430-446d-9960-3c05e7cbdb7e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:49:33.894612 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.894559 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8558844-6931-4eac-a660-380c1cd94130-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:49:33.896398 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.896369 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b6f4083-2430-446d-9960-3c05e7cbdb7e-kube-api-access-2jvpm" (OuterVolumeSpecName: "kube-api-access-2jvpm") pod "3b6f4083-2430-446d-9960-3c05e7cbdb7e" (UID: "3b6f4083-2430-446d-9960-3c05e7cbdb7e"). InnerVolumeSpecName "kube-api-access-2jvpm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:49:33.896398 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.896405 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8558844-6931-4eac-a660-380c1cd94130-kube-api-access-s7dv5" (OuterVolumeSpecName: "kube-api-access-s7dv5") pod "d8558844-6931-4eac-a660-380c1cd94130" (UID: "d8558844-6931-4eac-a660-380c1cd94130"). InnerVolumeSpecName "kube-api-access-s7dv5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:49:33.995380 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.995351 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7dv5\" (UniqueName: \"kubernetes.io/projected/d8558844-6931-4eac-a660-380c1cd94130-kube-api-access-s7dv5\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:49:33.995380 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.995381 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b6f4083-2430-446d-9960-3c05e7cbdb7e-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:49:33.995573 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:33.995391 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jvpm\" (UniqueName: \"kubernetes.io/projected/3b6f4083-2430-446d-9960-3c05e7cbdb7e-kube-api-access-2jvpm\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:49:34.062558 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.055840 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" event={"ID":"3b6f4083-2430-446d-9960-3c05e7cbdb7e","Type":"ContainerDied","Data":"952c3c7c8a125f75cc4b1f93f01f3e555ea096f98064c5ca96349f87d460f3de"} Jul 24 00:49:34.062558 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.055959 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" Jul 24 00:49:34.062558 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.055964 2560 scope.go:117] "RemoveContainer" containerID="952c3c7c8a125f75cc4b1f93f01f3e555ea096f98064c5ca96349f87d460f3de" Jul 24 00:49:34.062558 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.055813 2560 generic.go:356] "Generic (PLEG): container finished" podID="3b6f4083-2430-446d-9960-3c05e7cbdb7e" containerID="952c3c7c8a125f75cc4b1f93f01f3e555ea096f98064c5ca96349f87d460f3de" exitCode=0 Jul 24 00:49:34.062558 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.056786 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx" event={"ID":"3b6f4083-2430-446d-9960-3c05e7cbdb7e","Type":"ContainerDied","Data":"ecf6f6210cb5e788235563ab367fa431378e9b6c377c386f9a7ea1ae0853fb03"} Jul 24 00:49:34.064432 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.064398 2560 generic.go:356] "Generic (PLEG): container finished" podID="d8558844-6931-4eac-a660-380c1cd94130" containerID="7976a6e9d5aee9677c9ff8e3d8da76784f27bfe21469cc0a986ebd416ccfd4c3" exitCode=0 Jul 24 00:49:34.064536 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.064423 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" event={"ID":"d8558844-6931-4eac-a660-380c1cd94130","Type":"ContainerDied","Data":"7976a6e9d5aee9677c9ff8e3d8da76784f27bfe21469cc0a986ebd416ccfd4c3"} Jul 24 00:49:34.064536 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.064468 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" Jul 24 00:49:34.064536 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.064500 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq" event={"ID":"d8558844-6931-4eac-a660-380c1cd94130","Type":"ContainerDied","Data":"ee2a00dac6df867aa450f1144e23ebdea77a227ba89a1ef3f0881d739bf703eb"} Jul 24 00:49:34.072608 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.072585 2560 scope.go:117] "RemoveContainer" containerID="952c3c7c8a125f75cc4b1f93f01f3e555ea096f98064c5ca96349f87d460f3de" Jul 24 00:49:34.072968 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:49:34.072947 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"952c3c7c8a125f75cc4b1f93f01f3e555ea096f98064c5ca96349f87d460f3de\": container with ID starting with 952c3c7c8a125f75cc4b1f93f01f3e555ea096f98064c5ca96349f87d460f3de not found: ID does not exist" containerID="952c3c7c8a125f75cc4b1f93f01f3e555ea096f98064c5ca96349f87d460f3de" Jul 24 00:49:34.073052 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.072976 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"952c3c7c8a125f75cc4b1f93f01f3e555ea096f98064c5ca96349f87d460f3de"} err="failed to get container status \"952c3c7c8a125f75cc4b1f93f01f3e555ea096f98064c5ca96349f87d460f3de\": rpc error: code = NotFound desc = could not find container \"952c3c7c8a125f75cc4b1f93f01f3e555ea096f98064c5ca96349f87d460f3de\": container with ID starting with 952c3c7c8a125f75cc4b1f93f01f3e555ea096f98064c5ca96349f87d460f3de not found: ID does not exist" Jul 24 00:49:34.073052 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.072994 2560 scope.go:117] "RemoveContainer" containerID="7976a6e9d5aee9677c9ff8e3d8da76784f27bfe21469cc0a986ebd416ccfd4c3" Jul 24 00:49:34.081535 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.081514 2560 scope.go:117] "RemoveContainer" containerID="7976a6e9d5aee9677c9ff8e3d8da76784f27bfe21469cc0a986ebd416ccfd4c3" Jul 24 00:49:34.081768 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.081747 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx"] Jul 24 00:49:34.081934 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:49:34.081914 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7976a6e9d5aee9677c9ff8e3d8da76784f27bfe21469cc0a986ebd416ccfd4c3\": container with ID starting with 7976a6e9d5aee9677c9ff8e3d8da76784f27bfe21469cc0a986ebd416ccfd4c3 not found: ID does not exist" containerID="7976a6e9d5aee9677c9ff8e3d8da76784f27bfe21469cc0a986ebd416ccfd4c3" Jul 24 00:49:34.081977 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.081942 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7976a6e9d5aee9677c9ff8e3d8da76784f27bfe21469cc0a986ebd416ccfd4c3"} err="failed to get container status \"7976a6e9d5aee9677c9ff8e3d8da76784f27bfe21469cc0a986ebd416ccfd4c3\": rpc error: code = NotFound desc = could not find container \"7976a6e9d5aee9677c9ff8e3d8da76784f27bfe21469cc0a986ebd416ccfd4c3\": container with ID starting with 7976a6e9d5aee9677c9ff8e3d8da76784f27bfe21469cc0a986ebd416ccfd4c3 not found: ID does not exist" Jul 24 00:49:34.087100 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.087007 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-796b6a-74f855797d-vbjdx"] Jul 24 00:49:34.096331 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.096307 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq"] Jul 24 00:49:34.099513 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.099491 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-796b6a-84d47bd864-tjqqq"] Jul 24 00:49:34.348679 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.348578 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b6f4083-2430-446d-9960-3c05e7cbdb7e" path="/var/lib/kubelet/pods/3b6f4083-2430-446d-9960-3c05e7cbdb7e/volumes" Jul 24 00:49:34.349015 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:34.349001 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8558844-6931-4eac-a660-380c1cd94130" path="/var/lib/kubelet/pods/d8558844-6931-4eac-a660-380c1cd94130/volumes" Jul 24 00:49:37.852215 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:37.852186 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:38.264047 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:38.264015 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:44.384052 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:44.383994 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:44.510943 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:44.510890 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:44.698861 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:44.698762 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:49:44.930584 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:49:44.930535 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:00.522858 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.522825 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:00.546981 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.546950 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm"] Jul 24 00:50:00.547204 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.547177 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" podUID="9c652f42-aaca-4209-a6f6-f53386b8292f" containerName="payload-processing" containerID="cri-o://ef14d6abce1b311439af3d3cf07f39230629674c309d8f2520956ec03ae6362e" gracePeriod=30 Jul 24 00:50:00.558153 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.558100 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr"] Jul 24 00:50:00.558388 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.558339 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" podUID="daf1fb43-520d-4bad-b03a-3cb8c36f5b39" containerName="payload-pre-processing" containerID="cri-o://f351e05273f6823969ebdd47c2cfe493b9788cc88e7a70f27d109eac610745a0" gracePeriod=30 Jul 24 00:50:00.586491 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.586455 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:00.647651 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.647615 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:00.839473 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.839444 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" Jul 24 00:50:00.842591 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.842571 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" Jul 24 00:50:00.968353 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.968320 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxjtj\" (UniqueName: \"kubernetes.io/projected/daf1fb43-520d-4bad-b03a-3cb8c36f5b39-kube-api-access-wxjtj\") pod \"daf1fb43-520d-4bad-b03a-3cb8c36f5b39\" (UID: \"daf1fb43-520d-4bad-b03a-3cb8c36f5b39\") " Jul 24 00:50:00.968543 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.968369 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8ngs\" (UniqueName: \"kubernetes.io/projected/9c652f42-aaca-4209-a6f6-f53386b8292f-kube-api-access-b8ngs\") pod \"9c652f42-aaca-4209-a6f6-f53386b8292f\" (UID: \"9c652f42-aaca-4209-a6f6-f53386b8292f\") " Jul 24 00:50:00.968543 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.968399 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c652f42-aaca-4209-a6f6-f53386b8292f-plugins-config-volume\") pod \"9c652f42-aaca-4209-a6f6-f53386b8292f\" (UID: \"9c652f42-aaca-4209-a6f6-f53386b8292f\") " Jul 24 00:50:00.968543 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.968415 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daf1fb43-520d-4bad-b03a-3cb8c36f5b39-plugins-config-volume\") pod \"daf1fb43-520d-4bad-b03a-3cb8c36f5b39\" (UID: \"daf1fb43-520d-4bad-b03a-3cb8c36f5b39\") " Jul 24 00:50:00.968923 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.968895 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c652f42-aaca-4209-a6f6-f53386b8292f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9c652f42-aaca-4209-a6f6-f53386b8292f" (UID: "9c652f42-aaca-4209-a6f6-f53386b8292f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:50:00.968998 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.968921 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daf1fb43-520d-4bad-b03a-3cb8c36f5b39-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "daf1fb43-520d-4bad-b03a-3cb8c36f5b39" (UID: "daf1fb43-520d-4bad-b03a-3cb8c36f5b39"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:50:00.970614 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.970588 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daf1fb43-520d-4bad-b03a-3cb8c36f5b39-kube-api-access-wxjtj" (OuterVolumeSpecName: "kube-api-access-wxjtj") pod "daf1fb43-520d-4bad-b03a-3cb8c36f5b39" (UID: "daf1fb43-520d-4bad-b03a-3cb8c36f5b39"). InnerVolumeSpecName "kube-api-access-wxjtj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:50:00.970828 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:00.970798 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c652f42-aaca-4209-a6f6-f53386b8292f-kube-api-access-b8ngs" (OuterVolumeSpecName: "kube-api-access-b8ngs") pod "9c652f42-aaca-4209-a6f6-f53386b8292f" (UID: "9c652f42-aaca-4209-a6f6-f53386b8292f"). InnerVolumeSpecName "kube-api-access-b8ngs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:50:01.070082 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.069997 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxjtj\" (UniqueName: \"kubernetes.io/projected/daf1fb43-520d-4bad-b03a-3cb8c36f5b39-kube-api-access-wxjtj\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:50:01.070082 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.070025 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8ngs\" (UniqueName: \"kubernetes.io/projected/9c652f42-aaca-4209-a6f6-f53386b8292f-kube-api-access-b8ngs\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:50:01.070082 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.070034 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c652f42-aaca-4209-a6f6-f53386b8292f-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:50:01.070082 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.070046 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daf1fb43-520d-4bad-b03a-3cb8c36f5b39-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:50:01.214720 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.214689 2560 generic.go:356] "Generic (PLEG): container finished" podID="daf1fb43-520d-4bad-b03a-3cb8c36f5b39" containerID="f351e05273f6823969ebdd47c2cfe493b9788cc88e7a70f27d109eac610745a0" exitCode=0 Jul 24 00:50:01.214930 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.214736 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" Jul 24 00:50:01.214930 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.214777 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" event={"ID":"daf1fb43-520d-4bad-b03a-3cb8c36f5b39","Type":"ContainerDied","Data":"f351e05273f6823969ebdd47c2cfe493b9788cc88e7a70f27d109eac610745a0"} Jul 24 00:50:01.214930 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.214837 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr" event={"ID":"daf1fb43-520d-4bad-b03a-3cb8c36f5b39","Type":"ContainerDied","Data":"c14c9c3fdb4410c3619f97aaf722c261884ec9a08133cc591ff1d37425982e6e"} Jul 24 00:50:01.214930 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.214865 2560 scope.go:117] "RemoveContainer" containerID="f351e05273f6823969ebdd47c2cfe493b9788cc88e7a70f27d109eac610745a0" Jul 24 00:50:01.216123 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.216099 2560 generic.go:356] "Generic (PLEG): container finished" podID="9c652f42-aaca-4209-a6f6-f53386b8292f" containerID="ef14d6abce1b311439af3d3cf07f39230629674c309d8f2520956ec03ae6362e" exitCode=0 Jul 24 00:50:01.216237 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.216131 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" Jul 24 00:50:01.216237 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.216133 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" event={"ID":"9c652f42-aaca-4209-a6f6-f53386b8292f","Type":"ContainerDied","Data":"ef14d6abce1b311439af3d3cf07f39230629674c309d8f2520956ec03ae6362e"} Jul 24 00:50:01.216335 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.216237 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm" event={"ID":"9c652f42-aaca-4209-a6f6-f53386b8292f","Type":"ContainerDied","Data":"078a2cef546eb1ca9fe7d9b213ca49f793f2d58e0f045e94e584b06ee6741fa6"} Jul 24 00:50:01.224290 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.224270 2560 scope.go:117] "RemoveContainer" containerID="f351e05273f6823969ebdd47c2cfe493b9788cc88e7a70f27d109eac610745a0" Jul 24 00:50:01.224586 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:50:01.224567 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f351e05273f6823969ebdd47c2cfe493b9788cc88e7a70f27d109eac610745a0\": container with ID starting with f351e05273f6823969ebdd47c2cfe493b9788cc88e7a70f27d109eac610745a0 not found: ID does not exist" containerID="f351e05273f6823969ebdd47c2cfe493b9788cc88e7a70f27d109eac610745a0" Jul 24 00:50:01.224675 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.224593 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f351e05273f6823969ebdd47c2cfe493b9788cc88e7a70f27d109eac610745a0"} err="failed to get container status \"f351e05273f6823969ebdd47c2cfe493b9788cc88e7a70f27d109eac610745a0\": rpc error: code = NotFound desc = could not find container \"f351e05273f6823969ebdd47c2cfe493b9788cc88e7a70f27d109eac610745a0\": container with ID starting with f351e05273f6823969ebdd47c2cfe493b9788cc88e7a70f27d109eac610745a0 not found: ID does not exist" Jul 24 00:50:01.224675 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.224609 2560 scope.go:117] "RemoveContainer" containerID="ef14d6abce1b311439af3d3cf07f39230629674c309d8f2520956ec03ae6362e" Jul 24 00:50:01.232703 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.232680 2560 scope.go:117] "RemoveContainer" containerID="ef14d6abce1b311439af3d3cf07f39230629674c309d8f2520956ec03ae6362e" Jul 24 00:50:01.232944 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:50:01.232924 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef14d6abce1b311439af3d3cf07f39230629674c309d8f2520956ec03ae6362e\": container with ID starting with ef14d6abce1b311439af3d3cf07f39230629674c309d8f2520956ec03ae6362e not found: ID does not exist" containerID="ef14d6abce1b311439af3d3cf07f39230629674c309d8f2520956ec03ae6362e" Jul 24 00:50:01.233003 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.232950 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef14d6abce1b311439af3d3cf07f39230629674c309d8f2520956ec03ae6362e"} err="failed to get container status \"ef14d6abce1b311439af3d3cf07f39230629674c309d8f2520956ec03ae6362e\": rpc error: code = NotFound desc = could not find container \"ef14d6abce1b311439af3d3cf07f39230629674c309d8f2520956ec03ae6362e\": container with ID starting with ef14d6abce1b311439af3d3cf07f39230629674c309d8f2520956ec03ae6362e not found: ID does not exist" Jul 24 00:50:01.239691 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.239665 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr"] Jul 24 00:50:01.242990 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.242968 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a4c256-cb49b9584-6frfr"] Jul 24 00:50:01.253413 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.253386 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm"] Jul 24 00:50:01.261904 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:01.261882 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a4c256-6d7dbf4d8d-4tvkm"] Jul 24 00:50:02.356824 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:02.356794 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c652f42-aaca-4209-a6f6-f53386b8292f" path="/var/lib/kubelet/pods/9c652f42-aaca-4209-a6f6-f53386b8292f/volumes" Jul 24 00:50:02.357221 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:02.357169 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daf1fb43-520d-4bad-b03a-3cb8c36f5b39" path="/var/lib/kubelet/pods/daf1fb43-520d-4bad-b03a-3cb8c36f5b39/volumes" Jul 24 00:50:05.141950 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:05.141866 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:05.560244 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:05.560207 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:11.241416 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:11.241376 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:11.301123 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:11.301084 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:11.496516 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:11.496484 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb"] Jul 24 00:50:11.496749 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:11.496723 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" podUID="d7f81045-8cde-4571-b37e-356951980b2f" containerName="istio-proxy" containerID="cri-o://f7a0ecec002c42a3dddd3173e80cb93c189cfe9bf5f5144eebd2a713958e2719" gracePeriod=30 Jul 24 00:50:11.511988 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:11.511958 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:11.768623 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:11.768528 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:12.109752 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:12.109627 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:12.425292 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:12.425260 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:12.580156 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:12.580116 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:12.639280 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:12.639228 2560 patch_prober.go:28] interesting pod/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 00:50:12.639457 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:12.639309 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" podUID="d7f81045-8cde-4571-b37e-356951980b2f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 00:50:15.202614 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:15.202581 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:15.301063 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:15.301029 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:15.514867 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:15.514832 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:15.675395 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:15.675367 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:16.004382 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.004346 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:16.081093 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.081063 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:16.085607 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.085579 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v"] Jul 24 00:50:16.086043 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.086025 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b6f4083-2430-446d-9960-3c05e7cbdb7e" containerName="payload-processing" Jul 24 00:50:16.086043 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.086045 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b6f4083-2430-446d-9960-3c05e7cbdb7e" containerName="payload-processing" Jul 24 00:50:16.086223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.086068 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daf1fb43-520d-4bad-b03a-3cb8c36f5b39" containerName="payload-pre-processing" Jul 24 00:50:16.086223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.086075 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="daf1fb43-520d-4bad-b03a-3cb8c36f5b39" containerName="payload-pre-processing" Jul 24 00:50:16.086223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.086089 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c652f42-aaca-4209-a6f6-f53386b8292f" containerName="payload-processing" Jul 24 00:50:16.086223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.086095 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c652f42-aaca-4209-a6f6-f53386b8292f" containerName="payload-processing" Jul 24 00:50:16.086223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.086103 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8558844-6931-4eac-a660-380c1cd94130" containerName="payload-pre-processing" Jul 24 00:50:16.086223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.086108 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8558844-6931-4eac-a660-380c1cd94130" containerName="payload-pre-processing" Jul 24 00:50:16.086223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.086166 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="daf1fb43-520d-4bad-b03a-3cb8c36f5b39" containerName="payload-pre-processing" Jul 24 00:50:16.086223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.086174 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8558844-6931-4eac-a660-380c1cd94130" containerName="payload-pre-processing" Jul 24 00:50:16.086223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.086184 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b6f4083-2430-446d-9960-3c05e7cbdb7e" containerName="payload-processing" Jul 24 00:50:16.086223 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.086190 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c652f42-aaca-4209-a6f6-f53386b8292f" containerName="payload-processing" Jul 24 00:50:16.090888 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.090862 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" Jul 24 00:50:16.093672 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.093653 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-44952a\"" Jul 24 00:50:16.099681 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.099658 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v"] Jul 24 00:50:16.115714 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.115684 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn"] Jul 24 00:50:16.119183 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.119165 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" Jul 24 00:50:16.119562 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.119541 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v\" (UID: \"dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" Jul 24 00:50:16.119665 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.119625 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j7ff\" (UniqueName: \"kubernetes.io/projected/dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8-kube-api-access-2j7ff\") pod \"payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v\" (UID: \"dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" Jul 24 00:50:16.121809 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.121790 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-44952a-dockercfg-95txf\"" Jul 24 00:50:16.129132 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.129105 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn"] Jul 24 00:50:16.221100 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.221052 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v\" (UID: \"dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" Jul 24 00:50:16.221598 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.221393 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfzz7\" (UniqueName: \"kubernetes.io/projected/cd832f51-24ab-4f24-89cc-1332a4233318-kube-api-access-qfzz7\") pod \"payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn\" (UID: \"cd832f51-24ab-4f24-89cc-1332a4233318\") " pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" Jul 24 00:50:16.221598 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.221445 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j7ff\" (UniqueName: \"kubernetes.io/projected/dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8-kube-api-access-2j7ff\") pod \"payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v\" (UID: \"dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" Jul 24 00:50:16.221598 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.221505 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd832f51-24ab-4f24-89cc-1332a4233318-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn\" (UID: \"cd832f51-24ab-4f24-89cc-1332a4233318\") " pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" Jul 24 00:50:16.222083 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.222051 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v\" (UID: \"dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" Jul 24 00:50:16.231387 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.231358 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j7ff\" (UniqueName: \"kubernetes.io/projected/dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8-kube-api-access-2j7ff\") pod \"payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v\" (UID: \"dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" Jul 24 00:50:16.322395 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.322288 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfzz7\" (UniqueName: \"kubernetes.io/projected/cd832f51-24ab-4f24-89cc-1332a4233318-kube-api-access-qfzz7\") pod \"payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn\" (UID: \"cd832f51-24ab-4f24-89cc-1332a4233318\") " pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" Jul 24 00:50:16.322395 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.322372 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd832f51-24ab-4f24-89cc-1332a4233318-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn\" (UID: \"cd832f51-24ab-4f24-89cc-1332a4233318\") " pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" Jul 24 00:50:16.323175 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.323139 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd832f51-24ab-4f24-89cc-1332a4233318-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn\" (UID: \"cd832f51-24ab-4f24-89cc-1332a4233318\") " pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" Jul 24 00:50:16.332713 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.332687 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfzz7\" (UniqueName: \"kubernetes.io/projected/cd832f51-24ab-4f24-89cc-1332a4233318-kube-api-access-qfzz7\") pod \"payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn\" (UID: \"cd832f51-24ab-4f24-89cc-1332a4233318\") " pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" Jul 24 00:50:16.403632 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.403596 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" Jul 24 00:50:16.430766 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.430735 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" Jul 24 00:50:16.557523 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.557010 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v"] Jul 24 00:50:16.575003 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:50:16.574930 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbb4a18c_f4b7_4f89_aa79_a32fe254e9a8.slice/crio-b902aed322d22da50bf3cff8101e7d76fbae6a7ff23efc81232cecc7d8cbc142 WatchSource:0}: Error finding container b902aed322d22da50bf3cff8101e7d76fbae6a7ff23efc81232cecc7d8cbc142: Status 404 returned error can't find the container with id b902aed322d22da50bf3cff8101e7d76fbae6a7ff23efc81232cecc7d8cbc142 Jul 24 00:50:16.584692 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.584657 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn"] Jul 24 00:50:16.587938 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:50:16.587914 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd832f51_24ab_4f24_89cc_1332a4233318.slice/crio-d81d2fb43131ad87f45d60449897de4935da8f343b78b20dab5c8d6bf48b14e9 WatchSource:0}: Error finding container d81d2fb43131ad87f45d60449897de4935da8f343b78b20dab5c8d6bf48b14e9: Status 404 returned error can't find the container with id d81d2fb43131ad87f45d60449897de4935da8f343b78b20dab5c8d6bf48b14e9 Jul 24 00:50:16.754791 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.754766 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:50:16.828741 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.828613 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7f81045-8cde-4571-b37e-356951980b2f-istiod-ca-cert\") pod \"d7f81045-8cde-4571-b37e-356951980b2f\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " Jul 24 00:50:16.828741 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.828696 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7f81045-8cde-4571-b37e-356951980b2f-istio-podinfo\") pod \"d7f81045-8cde-4571-b37e-356951980b2f\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " Jul 24 00:50:16.828741 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.828727 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7f81045-8cde-4571-b37e-356951980b2f-istio-token\") pod \"d7f81045-8cde-4571-b37e-356951980b2f\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " Jul 24 00:50:16.829023 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.828773 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-workload-certs\") pod \"d7f81045-8cde-4571-b37e-356951980b2f\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " Jul 24 00:50:16.829023 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.828865 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-istio-envoy\") pod \"d7f81045-8cde-4571-b37e-356951980b2f\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " Jul 24 00:50:16.829023 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.828890 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-workload-socket\") pod \"d7f81045-8cde-4571-b37e-356951980b2f\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " Jul 24 00:50:16.829023 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.828937 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-istio-data\") pod \"d7f81045-8cde-4571-b37e-356951980b2f\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " Jul 24 00:50:16.829023 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.828962 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qm47\" (UniqueName: \"kubernetes.io/projected/d7f81045-8cde-4571-b37e-356951980b2f-kube-api-access-7qm47\") pod \"d7f81045-8cde-4571-b37e-356951980b2f\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " Jul 24 00:50:16.829023 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.828992 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-credential-socket\") pod \"d7f81045-8cde-4571-b37e-356951980b2f\" (UID: \"d7f81045-8cde-4571-b37e-356951980b2f\") " Jul 24 00:50:16.829317 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.829014 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7f81045-8cde-4571-b37e-356951980b2f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7f81045-8cde-4571-b37e-356951980b2f" (UID: "d7f81045-8cde-4571-b37e-356951980b2f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:50:16.829317 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.829122 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7f81045-8cde-4571-b37e-356951980b2f" (UID: "d7f81045-8cde-4571-b37e-356951980b2f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:50:16.829431 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.829332 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-workload-certs\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:50:16.829431 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.829354 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7f81045-8cde-4571-b37e-356951980b2f-istiod-ca-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:50:16.829431 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.829354 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7f81045-8cde-4571-b37e-356951980b2f" (UID: "d7f81045-8cde-4571-b37e-356951980b2f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:50:16.829578 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.829546 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7f81045-8cde-4571-b37e-356951980b2f" (UID: "d7f81045-8cde-4571-b37e-356951980b2f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:50:16.829889 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.829857 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7f81045-8cde-4571-b37e-356951980b2f" (UID: "d7f81045-8cde-4571-b37e-356951980b2f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:50:16.831001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.830977 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7f81045-8cde-4571-b37e-356951980b2f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7f81045-8cde-4571-b37e-356951980b2f" (UID: "d7f81045-8cde-4571-b37e-356951980b2f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 00:50:16.831083 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.831063 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7f81045-8cde-4571-b37e-356951980b2f-kube-api-access-7qm47" (OuterVolumeSpecName: "kube-api-access-7qm47") pod "d7f81045-8cde-4571-b37e-356951980b2f" (UID: "d7f81045-8cde-4571-b37e-356951980b2f"). InnerVolumeSpecName "kube-api-access-7qm47". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:50:16.831360 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.831341 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7f81045-8cde-4571-b37e-356951980b2f-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7f81045-8cde-4571-b37e-356951980b2f" (UID: "d7f81045-8cde-4571-b37e-356951980b2f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:50:16.831360 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.831348 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7f81045-8cde-4571-b37e-356951980b2f" (UID: "d7f81045-8cde-4571-b37e-356951980b2f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:50:16.930209 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.930162 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-istio-envoy\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:50:16.930209 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.930200 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-workload-socket\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:50:16.930209 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.930220 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-istio-data\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:50:16.930471 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.930233 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qm47\" (UniqueName: \"kubernetes.io/projected/d7f81045-8cde-4571-b37e-356951980b2f-kube-api-access-7qm47\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:50:16.930471 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.930248 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f81045-8cde-4571-b37e-356951980b2f-credential-socket\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:50:16.930471 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.930260 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7f81045-8cde-4571-b37e-356951980b2f-istio-podinfo\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:50:16.930471 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:16.930271 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7f81045-8cde-4571-b37e-356951980b2f-istio-token\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:50:17.307220 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.307177 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" event={"ID":"cd832f51-24ab-4f24-89cc-1332a4233318","Type":"ContainerStarted","Data":"7b003074d2d34b6fbe146a6fe599002eaacefae79dbe8d3d2054c615c0105273"} Jul 24 00:50:17.307741 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.307288 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" Jul 24 00:50:17.307741 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.307315 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" event={"ID":"cd832f51-24ab-4f24-89cc-1332a4233318","Type":"ContainerStarted","Data":"d81d2fb43131ad87f45d60449897de4935da8f343b78b20dab5c8d6bf48b14e9"} Jul 24 00:50:17.308771 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.308738 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" event={"ID":"dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8","Type":"ContainerStarted","Data":"46efae517e36d931c08c037ff80aa920b3975b68701754dde71d0c58f1110acc"} Jul 24 00:50:17.308904 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.308789 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" event={"ID":"dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8","Type":"ContainerStarted","Data":"b902aed322d22da50bf3cff8101e7d76fbae6a7ff23efc81232cecc7d8cbc142"} Jul 24 00:50:17.308904 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.308847 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" Jul 24 00:50:17.310447 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.310418 2560 generic.go:356] "Generic (PLEG): container finished" podID="d7f81045-8cde-4571-b37e-356951980b2f" containerID="f7a0ecec002c42a3dddd3173e80cb93c189cfe9bf5f5144eebd2a713958e2719" exitCode=0 Jul 24 00:50:17.310554 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.310444 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" event={"ID":"d7f81045-8cde-4571-b37e-356951980b2f","Type":"ContainerDied","Data":"f7a0ecec002c42a3dddd3173e80cb93c189cfe9bf5f5144eebd2a713958e2719"} Jul 24 00:50:17.310554 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.310481 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" Jul 24 00:50:17.310554 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.310497 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb" event={"ID":"d7f81045-8cde-4571-b37e-356951980b2f","Type":"ContainerDied","Data":"8ebc2e53ccb1493eb71667d7c94fc987a7f041bacceee456185000782cec205b"} Jul 24 00:50:17.310554 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.310519 2560 scope.go:117] "RemoveContainer" containerID="f7a0ecec002c42a3dddd3173e80cb93c189cfe9bf5f5144eebd2a713958e2719" Jul 24 00:50:17.319702 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.319684 2560 scope.go:117] "RemoveContainer" containerID="e3b75badc6bf665a64f52e77c9d2b71449bcca28cd17d9021d3ae0f1c74f6a5e" Jul 24 00:50:17.329210 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.329151 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" podStartSLOduration=1.32913082 podStartE2EDuration="1.32913082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:50:17.325381422 +0000 UTC m=+2345.930695736" watchObservedRunningTime="2026-07-24 00:50:17.32913082 +0000 UTC m=+2345.934445150" Jul 24 00:50:17.329964 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.329947 2560 scope.go:117] "RemoveContainer" containerID="f7a0ecec002c42a3dddd3173e80cb93c189cfe9bf5f5144eebd2a713958e2719" Jul 24 00:50:17.330257 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:50:17.330224 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7a0ecec002c42a3dddd3173e80cb93c189cfe9bf5f5144eebd2a713958e2719\": container with ID starting with f7a0ecec002c42a3dddd3173e80cb93c189cfe9bf5f5144eebd2a713958e2719 not found: ID does not exist" containerID="f7a0ecec002c42a3dddd3173e80cb93c189cfe9bf5f5144eebd2a713958e2719" Jul 24 00:50:17.330326 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.330267 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7a0ecec002c42a3dddd3173e80cb93c189cfe9bf5f5144eebd2a713958e2719"} err="failed to get container status \"f7a0ecec002c42a3dddd3173e80cb93c189cfe9bf5f5144eebd2a713958e2719\": rpc error: code = NotFound desc = could not find container \"f7a0ecec002c42a3dddd3173e80cb93c189cfe9bf5f5144eebd2a713958e2719\": container with ID starting with f7a0ecec002c42a3dddd3173e80cb93c189cfe9bf5f5144eebd2a713958e2719 not found: ID does not exist" Jul 24 00:50:17.330326 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.330291 2560 scope.go:117] "RemoveContainer" containerID="e3b75badc6bf665a64f52e77c9d2b71449bcca28cd17d9021d3ae0f1c74f6a5e" Jul 24 00:50:17.330600 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:50:17.330578 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3b75badc6bf665a64f52e77c9d2b71449bcca28cd17d9021d3ae0f1c74f6a5e\": container with ID starting with e3b75badc6bf665a64f52e77c9d2b71449bcca28cd17d9021d3ae0f1c74f6a5e not found: ID does not exist" containerID="e3b75badc6bf665a64f52e77c9d2b71449bcca28cd17d9021d3ae0f1c74f6a5e" Jul 24 00:50:17.330714 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.330629 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3b75badc6bf665a64f52e77c9d2b71449bcca28cd17d9021d3ae0f1c74f6a5e"} err="failed to get container status \"e3b75badc6bf665a64f52e77c9d2b71449bcca28cd17d9021d3ae0f1c74f6a5e\": rpc error: code = NotFound desc = could not find container \"e3b75badc6bf665a64f52e77c9d2b71449bcca28cd17d9021d3ae0f1c74f6a5e\": container with ID starting with e3b75badc6bf665a64f52e77c9d2b71449bcca28cd17d9021d3ae0f1c74f6a5e not found: ID does not exist" Jul 24 00:50:17.340038 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.340011 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb"] Jul 24 00:50:17.347038 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.347015 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a4c256-openshift-default-88498bd5b-vgglb"] Jul 24 00:50:17.360521 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:17.360469 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" podStartSLOduration=1.360451278 podStartE2EDuration="1.360451278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:50:17.359729618 +0000 UTC m=+2345.965043934" watchObservedRunningTime="2026-07-24 00:50:17.360451278 +0000 UTC m=+2345.965765595" Jul 24 00:50:18.348506 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:18.348471 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7f81045-8cde-4571-b37e-356951980b2f" path="/var/lib/kubelet/pods/d7f81045-8cde-4571-b37e-356951980b2f/volumes" Jul 24 00:50:19.716936 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:19.716904 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:20.082404 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.082338 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:20.366675 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.366585 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v"] Jul 24 00:50:20.367027 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.367014 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7f81045-8cde-4571-b37e-356951980b2f" containerName="istio-proxy" Jul 24 00:50:20.367077 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.367028 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7f81045-8cde-4571-b37e-356951980b2f" containerName="istio-proxy" Jul 24 00:50:20.367110 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.367101 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7f81045-8cde-4571-b37e-356951980b2f" containerName="istio-proxy" Jul 24 00:50:20.367146 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.367109 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7f81045-8cde-4571-b37e-356951980b2f" containerName="istio-proxy" Jul 24 00:50:20.371751 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.371706 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.374313 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.374293 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-91a75b-openshift-default-dockercfg-qsxdm\"" Jul 24 00:50:20.381912 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.381875 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v"] Jul 24 00:50:20.390993 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.390959 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:20.462749 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.462713 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:20.463829 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.463807 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hptkr\" (UniqueName: \"kubernetes.io/projected/a3007cad-2bda-4101-af9d-9bf95d19c099-kube-api-access-hptkr\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.463903 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.463840 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-credential-socket\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.463974 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.463957 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-workload-socket\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.464033 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.464019 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-workload-certs\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.464086 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.464044 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-data\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.464086 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.464065 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-podinfo\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.464170 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.464100 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-envoy\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.464170 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.464122 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-token\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.464170 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.464144 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3007cad-2bda-4101-af9d-9bf95d19c099-istiod-ca-cert\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.564750 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.564719 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-workload-socket\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.564935 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.564777 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-workload-certs\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.564935 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.564798 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-data\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.564935 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.564816 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-podinfo\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.564935 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.564841 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-envoy\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.564935 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.564865 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-token\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.564935 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.564889 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3007cad-2bda-4101-af9d-9bf95d19c099-istiod-ca-cert\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.565159 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.565082 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hptkr\" (UniqueName: \"kubernetes.io/projected/a3007cad-2bda-4101-af9d-9bf95d19c099-kube-api-access-hptkr\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.565159 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.565126 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-credential-socket\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.565314 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.565151 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-workload-socket\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.565967 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.565751 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-workload-certs\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.565967 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.565886 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-data\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.566408 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.566383 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3007cad-2bda-4101-af9d-9bf95d19c099-istiod-ca-cert\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.566568 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.566405 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-credential-socket\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.571749 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.568684 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-envoy\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.576769 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.576386 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-podinfo\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.578143 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.578119 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-token\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.578356 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.578335 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hptkr\" (UniqueName: \"kubernetes.io/projected/a3007cad-2bda-4101-af9d-9bf95d19c099-kube-api-access-hptkr\") pod \"e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.686465 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.686363 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:20.826156 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.826125 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v"] Jul 24 00:50:20.828149 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:50:20.828123 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3007cad_2bda_4101_af9d_9bf95d19c099.slice/crio-46f5363d610ead8845d9860dc949bbb219acae609ce7a584cee470689b0c99ef WatchSource:0}: Error finding container 46f5363d610ead8845d9860dc949bbb219acae609ce7a584cee470689b0c99ef: Status 404 returned error can't find the container with id 46f5363d610ead8845d9860dc949bbb219acae609ce7a584cee470689b0c99ef Jul 24 00:50:20.830998 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.830966 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:50:20.831095 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.831030 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:50:20.831095 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:20.831074 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:50:21.337682 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:21.337622 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" event={"ID":"a3007cad-2bda-4101-af9d-9bf95d19c099","Type":"ContainerStarted","Data":"e1c1663be9217217bba842391b3705daad24eebf785000d8fa7e1b07ead1535b"} Jul 24 00:50:21.337874 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:21.337708 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" event={"ID":"a3007cad-2bda-4101-af9d-9bf95d19c099","Type":"ContainerStarted","Data":"46f5363d610ead8845d9860dc949bbb219acae609ce7a584cee470689b0c99ef"} Jul 24 00:50:21.357973 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:21.357921 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" podStartSLOduration=1.357904228 podStartE2EDuration="1.357904228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:50:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:50:21.357298489 +0000 UTC m=+2349.962612876" watchObservedRunningTime="2026-07-24 00:50:21.357904228 +0000 UTC m=+2349.963218543" Jul 24 00:50:21.687253 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:21.687153 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:21.692714 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:21.692627 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:22.347958 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:22.347929 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:22.348332 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:22.347999 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:23.393794 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:23.393758 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:23.497221 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:23.497187 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:23.631608 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:23.631578 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:23.793324 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:23.793276 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:24.173304 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.173219 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:24.273288 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.273257 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:24.315483 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.315446 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn"] Jul 24 00:50:24.315972 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.315955 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7f81045-8cde-4571-b37e-356951980b2f" containerName="istio-proxy" Jul 24 00:50:24.315972 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.315974 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7f81045-8cde-4571-b37e-356951980b2f" containerName="istio-proxy" Jul 24 00:50:24.319239 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.319221 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" Jul 24 00:50:24.321905 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.321878 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-91a75b-dockercfg-rlq6s\"" Jul 24 00:50:24.322128 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.322112 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-91a75b\"" Jul 24 00:50:24.329879 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.329850 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn"] Jul 24 00:50:24.401097 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.401057 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfghm\" (UniqueName: \"kubernetes.io/projected/93235e8c-eaf6-4484-a413-c0bd7f800f1f-kube-api-access-bfghm\") pod \"payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn\" (UID: \"93235e8c-eaf6-4484-a413-c0bd7f800f1f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" Jul 24 00:50:24.401727 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.401314 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93235e8c-eaf6-4484-a413-c0bd7f800f1f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn\" (UID: \"93235e8c-eaf6-4484-a413-c0bd7f800f1f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" Jul 24 00:50:24.502649 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.502611 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93235e8c-eaf6-4484-a413-c0bd7f800f1f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn\" (UID: \"93235e8c-eaf6-4484-a413-c0bd7f800f1f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" Jul 24 00:50:24.502875 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.502734 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfghm\" (UniqueName: \"kubernetes.io/projected/93235e8c-eaf6-4484-a413-c0bd7f800f1f-kube-api-access-bfghm\") pod \"payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn\" (UID: \"93235e8c-eaf6-4484-a413-c0bd7f800f1f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" Jul 24 00:50:24.503380 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.503352 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93235e8c-eaf6-4484-a413-c0bd7f800f1f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn\" (UID: \"93235e8c-eaf6-4484-a413-c0bd7f800f1f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" Jul 24 00:50:24.511776 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.511751 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfghm\" (UniqueName: \"kubernetes.io/projected/93235e8c-eaf6-4484-a413-c0bd7f800f1f-kube-api-access-bfghm\") pod \"payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn\" (UID: \"93235e8c-eaf6-4484-a413-c0bd7f800f1f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" Jul 24 00:50:24.631907 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.631865 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" Jul 24 00:50:24.773373 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:24.773343 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn"] Jul 24 00:50:24.775343 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:50:24.775310 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93235e8c_eaf6_4484_a413_c0bd7f800f1f.slice/crio-44b8cecae6151f42cd558faf44077a37ea9570d35384d6a4735fd092c9794e9c WatchSource:0}: Error finding container 44b8cecae6151f42cd558faf44077a37ea9570d35384d6a4735fd092c9794e9c: Status 404 returned error can't find the container with id 44b8cecae6151f42cd558faf44077a37ea9570d35384d6a4735fd092c9794e9c Jul 24 00:50:25.362494 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:25.362458 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" event={"ID":"93235e8c-eaf6-4484-a413-c0bd7f800f1f","Type":"ContainerStarted","Data":"2516aa50eb42f39e683c7c48d013770068d4a1eef125492733c8eea73b3ecb20"} Jul 24 00:50:25.362710 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:25.362522 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" Jul 24 00:50:25.362710 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:25.362543 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" event={"ID":"93235e8c-eaf6-4484-a413-c0bd7f800f1f","Type":"ContainerStarted","Data":"44b8cecae6151f42cd558faf44077a37ea9570d35384d6a4735fd092c9794e9c"} Jul 24 00:50:25.383315 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:25.383254 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" podStartSLOduration=1.383237375 podStartE2EDuration="1.383237375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:50:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:50:25.381785747 +0000 UTC m=+2353.987100064" watchObservedRunningTime="2026-07-24 00:50:25.383237375 +0000 UTC m=+2353.988551691" Jul 24 00:50:27.815251 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:27.815212 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:28.323313 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:28.323280 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" Jul 24 00:50:28.323496 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:28.323353 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" Jul 24 00:50:36.369845 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:36.369808 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" Jul 24 00:50:37.350551 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:37.350512 2560 patch_prober.go:28] interesting pod/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 24 00:50:37.350799 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:37.350579 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" podUID="a3007cad-2bda-4101-af9d-9bf95d19c099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 24 00:50:37.438401 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:37.438365 2560 generic.go:356] "Generic (PLEG): container finished" podID="a3007cad-2bda-4101-af9d-9bf95d19c099" containerID="e1c1663be9217217bba842391b3705daad24eebf785000d8fa7e1b07ead1535b" exitCode=0 Jul 24 00:50:37.438927 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:37.438442 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" event={"ID":"a3007cad-2bda-4101-af9d-9bf95d19c099","Type":"ContainerDied","Data":"e1c1663be9217217bba842391b3705daad24eebf785000d8fa7e1b07ead1535b"} Jul 24 00:50:37.439042 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:37.439024 2560 scope.go:117] "RemoveContainer" containerID="e1c1663be9217217bba842391b3705daad24eebf785000d8fa7e1b07ead1535b" Jul 24 00:50:37.440186 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:37.440151 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:50:37.440292 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:37.440236 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:50:37.440292 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:37.440283 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:50:38.452027 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:38.451978 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" event={"ID":"a3007cad-2bda-4101-af9d-9bf95d19c099","Type":"ContainerStarted","Data":"a31834ccafad6d0b51ac5046be31f6ed422688288a21fabda5933017b7127c14"} Jul 24 00:50:38.551192 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:38.551155 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:50:38.687624 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:38.687577 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:38.687857 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:38.687730 2560 patch_prober.go:28] interesting pod/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 24 00:50:38.687857 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:38.687786 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" podUID="a3007cad-2bda-4101-af9d-9bf95d19c099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 24 00:50:39.686886 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:39.686844 2560 patch_prober.go:28] interesting pod/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 24 00:50:39.687512 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:39.687476 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" podUID="a3007cad-2bda-4101-af9d-9bf95d19c099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 24 00:50:40.687056 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:40.687011 2560 patch_prober.go:28] interesting pod/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 24 00:50:40.687576 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:40.687081 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" podUID="a3007cad-2bda-4101-af9d-9bf95d19c099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 24 00:50:41.687978 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:41.687935 2560 patch_prober.go:28] interesting pod/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 24 00:50:41.689042 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:41.688581 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" podUID="a3007cad-2bda-4101-af9d-9bf95d19c099" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 24 00:50:42.691406 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:42.691375 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:42.691900 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:42.691675 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:42.692525 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:42.692507 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 00:50:52.458557 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:50:52.458516 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:51:09.556127 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:51:09.556095 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:52:03.445707 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:52:03.445623 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:52:28.652440 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:52:28.652399 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:53:10.762987 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:53:10.762903 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:53:11.936092 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:53:11.936056 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:53:25.227696 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:53:25.227662 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:54:04.675329 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:54:04.675291 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:54:49.357947 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:54:49.357866 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:54:50.336999 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:54:50.336962 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:55:12.074651 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:55:12.074613 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:55:50.263618 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:55:50.263581 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:56:11.433972 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:56:11.433882 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:56:11.536015 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:56:11.535978 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:56:28.703245 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:56:28.703209 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:56:31.938881 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:56:31.938842 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:56:40.525793 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:56:40.525760 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:56:41.368241 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:56:41.368202 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:01.314225 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.314178 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:01.626403 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.626318 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt"] Jul 24 00:57:01.630076 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.630059 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.632749 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.632729 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-193ff5-openshift-default-dockercfg-kznkv\"" Jul 24 00:57:01.642167 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.642126 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istiod-ca-cert\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.642309 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.642169 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-workload-certs\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.642309 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.642230 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-workload-socket\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.642309 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.642261 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-data\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.642309 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.642291 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-podinfo\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.642309 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.642306 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-token\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.642561 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.642338 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-envoy\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.642561 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.642370 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-credential-socket\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.642561 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.642401 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvvcw\" (UniqueName: \"kubernetes.io/projected/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-kube-api-access-nvvcw\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.643395 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.643369 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt"] Jul 24 00:57:01.652715 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.652684 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:01.743322 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.743288 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istiod-ca-cert\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.743322 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.743325 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-workload-certs\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.743568 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.743357 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-workload-socket\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.743568 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.743379 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-data\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.743568 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.743417 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-podinfo\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.743568 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.743441 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-token\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.743568 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.743473 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-envoy\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.743568 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.743504 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-credential-socket\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.743568 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.743536 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvvcw\" (UniqueName: \"kubernetes.io/projected/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-kube-api-access-nvvcw\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.743972 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.743791 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-workload-certs\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.743972 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.743814 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-data\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.743972 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.743906 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-workload-socket\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.744335 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.743990 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-credential-socket\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.744335 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.744197 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istiod-ca-cert\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.745816 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.745794 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-envoy\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.746048 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.746031 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-podinfo\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.752515 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.752485 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-token\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.752734 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.752716 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvvcw\" (UniqueName: \"kubernetes.io/projected/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-kube-api-access-nvvcw\") pod \"e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:01.929487 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.929398 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:01.940519 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:01.940491 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:02.094276 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:02.094251 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt"] Jul 24 00:57:02.096464 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:57:02.096436 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedf2518a_9c33_4160_ad0f_ffbafc54c2d9.slice/crio-fc492d81415f15d4dce8a849aa7da3897cd224b4aa5915be8f064dbcdf2c7fb4 WatchSource:0}: Error finding container fc492d81415f15d4dce8a849aa7da3897cd224b4aa5915be8f064dbcdf2c7fb4: Status 404 returned error can't find the container with id fc492d81415f15d4dce8a849aa7da3897cd224b4aa5915be8f064dbcdf2c7fb4 Jul 24 00:57:02.098563 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:02.098529 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:57:02.098696 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:02.098594 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:57:02.098696 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:02.098624 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:57:02.457802 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:02.457766 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" event={"ID":"edf2518a-9c33-4160-ad0f-ffbafc54c2d9","Type":"ContainerStarted","Data":"9a4cdb1bd17594400ed07d0952e8a02d6694a36b2f6d504bbf9d0d66559ca0b7"} Jul 24 00:57:02.458298 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:02.457818 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" event={"ID":"edf2518a-9c33-4160-ad0f-ffbafc54c2d9","Type":"ContainerStarted","Data":"fc492d81415f15d4dce8a849aa7da3897cd224b4aa5915be8f064dbcdf2c7fb4"} Jul 24 00:57:02.477767 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:02.477716 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" podStartSLOduration=1.477701201 podStartE2EDuration="1.477701201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:57:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:57:02.475809472 +0000 UTC m=+2751.081123788" watchObservedRunningTime="2026-07-24 00:57:02.477701201 +0000 UTC m=+2751.083015517" Jul 24 00:57:02.940669 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:02.940611 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:03.946333 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:03.946306 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:04.461417 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:04.461383 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:04.469025 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:04.468996 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:04.469933 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:04.469912 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:04.558055 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:04.558018 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:04.622321 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:04.622258 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:04.840030 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:04.839996 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:05.052585 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:05.052553 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:05.152635 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:05.152542 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:09.002898 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:09.002865 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:19.968329 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:19.968289 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:20.291264 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:20.291227 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:20.391194 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:20.391163 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:20.510816 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:20.510771 2560 patch_prober.go:28] interesting pod/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 00:57:20.511022 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:20.510839 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" podUID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 00:57:21.557428 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:21.557394 2560 generic.go:356] "Generic (PLEG): container finished" podID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerID="9a4cdb1bd17594400ed07d0952e8a02d6694a36b2f6d504bbf9d0d66559ca0b7" exitCode=0 Jul 24 00:57:21.557878 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:21.557467 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" event={"ID":"edf2518a-9c33-4160-ad0f-ffbafc54c2d9","Type":"ContainerDied","Data":"9a4cdb1bd17594400ed07d0952e8a02d6694a36b2f6d504bbf9d0d66559ca0b7"} Jul 24 00:57:21.557878 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:21.557862 2560 scope.go:117] "RemoveContainer" containerID="9a4cdb1bd17594400ed07d0952e8a02d6694a36b2f6d504bbf9d0d66559ca0b7" Jul 24 00:57:21.558916 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:21.558890 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:57:21.558990 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:21.558955 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:57:21.559026 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:21.558989 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:57:22.564182 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:22.564139 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" event={"ID":"edf2518a-9c33-4160-ad0f-ffbafc54c2d9","Type":"ContainerStarted","Data":"45321e6485dddb1a7561dfb42c535c6875f688fb201deb9e14191b3f620f82a4"} Jul 24 00:57:22.940777 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:22.940681 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:22.940948 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:22.940898 2560 patch_prober.go:28] interesting pod/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 24 00:57:22.941011 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:22.940945 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" podUID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 24 00:57:23.110582 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:23.110546 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:23.208312 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:23.208231 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:23.391776 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:23.391739 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:23.941547 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:23.941510 2560 patch_prober.go:28] interesting pod/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 24 00:57:23.941975 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:23.941563 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" podUID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 24 00:57:23.969784 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:23.969752 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:24.167815 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.167782 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:24.268382 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.268354 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:24.287078 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.287035 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg"] Jul 24 00:57:24.296840 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.296811 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" Jul 24 00:57:24.303072 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.303044 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-4fa1d5-dockercfg-dsgtp\"" Jul 24 00:57:24.303237 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.303130 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-4fa1d5\"" Jul 24 00:57:24.303966 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.303938 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg"] Jul 24 00:57:24.351213 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.351181 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x"] Jul 24 00:57:24.355013 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.354997 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" Jul 24 00:57:24.365114 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.365087 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x"] Jul 24 00:57:24.408664 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.408255 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpzsb\" (UniqueName: \"kubernetes.io/projected/83dd244e-6328-49e0-862d-29492c8d1121-kube-api-access-qpzsb\") pod \"payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg\" (UID: \"83dd244e-6328-49e0-862d-29492c8d1121\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" Jul 24 00:57:24.408664 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.408308 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83dd244e-6328-49e0-862d-29492c8d1121-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg\" (UID: \"83dd244e-6328-49e0-862d-29492c8d1121\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" Jul 24 00:57:24.509701 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.509650 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpzsb\" (UniqueName: \"kubernetes.io/projected/83dd244e-6328-49e0-862d-29492c8d1121-kube-api-access-qpzsb\") pod \"payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg\" (UID: \"83dd244e-6328-49e0-862d-29492c8d1121\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" Jul 24 00:57:24.509885 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.509714 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83dd244e-6328-49e0-862d-29492c8d1121-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg\" (UID: \"83dd244e-6328-49e0-862d-29492c8d1121\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" Jul 24 00:57:24.509885 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.509757 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27nh7\" (UniqueName: \"kubernetes.io/projected/323ab7ae-1d94-42cc-bf67-914ba3033fde-kube-api-access-27nh7\") pod \"payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x\" (UID: \"323ab7ae-1d94-42cc-bf67-914ba3033fde\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" Jul 24 00:57:24.509885 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.509845 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/323ab7ae-1d94-42cc-bf67-914ba3033fde-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x\" (UID: \"323ab7ae-1d94-42cc-bf67-914ba3033fde\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" Jul 24 00:57:24.510480 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.510460 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83dd244e-6328-49e0-862d-29492c8d1121-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg\" (UID: \"83dd244e-6328-49e0-862d-29492c8d1121\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" Jul 24 00:57:24.517803 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.517773 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpzsb\" (UniqueName: \"kubernetes.io/projected/83dd244e-6328-49e0-862d-29492c8d1121-kube-api-access-qpzsb\") pod \"payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg\" (UID: \"83dd244e-6328-49e0-862d-29492c8d1121\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" Jul 24 00:57:24.608977 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.608880 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" Jul 24 00:57:24.611341 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.611317 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27nh7\" (UniqueName: \"kubernetes.io/projected/323ab7ae-1d94-42cc-bf67-914ba3033fde-kube-api-access-27nh7\") pod \"payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x\" (UID: \"323ab7ae-1d94-42cc-bf67-914ba3033fde\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" Jul 24 00:57:24.611413 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.611383 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/323ab7ae-1d94-42cc-bf67-914ba3033fde-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x\" (UID: \"323ab7ae-1d94-42cc-bf67-914ba3033fde\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" Jul 24 00:57:24.612230 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.612207 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/323ab7ae-1d94-42cc-bf67-914ba3033fde-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x\" (UID: \"323ab7ae-1d94-42cc-bf67-914ba3033fde\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" Jul 24 00:57:24.620295 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.620267 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27nh7\" (UniqueName: \"kubernetes.io/projected/323ab7ae-1d94-42cc-bf67-914ba3033fde-kube-api-access-27nh7\") pod \"payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x\" (UID: \"323ab7ae-1d94-42cc-bf67-914ba3033fde\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" Jul 24 00:57:24.667088 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.667060 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" Jul 24 00:57:24.890721 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.883735 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x"] Jul 24 00:57:24.894714 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:57:24.894678 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod323ab7ae_1d94_42cc_bf67_914ba3033fde.slice/crio-1ff09ca547f9839cb457bdc0b3dbd8c62c916e0f1f7b4efa5009503764797665 WatchSource:0}: Error finding container 1ff09ca547f9839cb457bdc0b3dbd8c62c916e0f1f7b4efa5009503764797665: Status 404 returned error can't find the container with id 1ff09ca547f9839cb457bdc0b3dbd8c62c916e0f1f7b4efa5009503764797665 Jul 24 00:57:24.941653 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.941619 2560 patch_prober.go:28] interesting pod/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 24 00:57:24.941990 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.941682 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" podUID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 24 00:57:24.990811 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:24.990601 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg"] Jul 24 00:57:24.997837 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:57:24.997794 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83dd244e_6328_49e0_862d_29492c8d1121.slice/crio-2b99d217668d01bbae66c5ccb5779579ce54ba97ff0908b23e5591f14b2ff501 WatchSource:0}: Error finding container 2b99d217668d01bbae66c5ccb5779579ce54ba97ff0908b23e5591f14b2ff501: Status 404 returned error can't find the container with id 2b99d217668d01bbae66c5ccb5779579ce54ba97ff0908b23e5591f14b2ff501 Jul 24 00:57:25.589901 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:25.588925 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" event={"ID":"323ab7ae-1d94-42cc-bf67-914ba3033fde","Type":"ContainerStarted","Data":"668978be11be9834ad7359e7b5b205c2c20e5690437c5593804b57a7e3e35a70"} Jul 24 00:57:25.589901 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:25.588998 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" event={"ID":"323ab7ae-1d94-42cc-bf67-914ba3033fde","Type":"ContainerStarted","Data":"1ff09ca547f9839cb457bdc0b3dbd8c62c916e0f1f7b4efa5009503764797665"} Jul 24 00:57:25.589901 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:25.589843 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" Jul 24 00:57:25.592509 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:25.592482 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" event={"ID":"83dd244e-6328-49e0-862d-29492c8d1121","Type":"ContainerStarted","Data":"3447ad77e56c0d00fc98deb3c559f8d2d00920d7eb16647d2a60553ed3b13af4"} Jul 24 00:57:25.592711 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:25.592692 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" event={"ID":"83dd244e-6328-49e0-862d-29492c8d1121","Type":"ContainerStarted","Data":"2b99d217668d01bbae66c5ccb5779579ce54ba97ff0908b23e5591f14b2ff501"} Jul 24 00:57:25.594666 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:25.593309 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" Jul 24 00:57:25.607407 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:25.607358 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" podStartSLOduration=1.6073413090000002 podStartE2EDuration="1.607341309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:57:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:57:25.604683057 +0000 UTC m=+2774.209997375" watchObservedRunningTime="2026-07-24 00:57:25.607341309 +0000 UTC m=+2774.212655646" Jul 24 00:57:25.623302 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:25.623256 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" podStartSLOduration=1.623239823 podStartE2EDuration="1.623239823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:57:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:57:25.621245238 +0000 UTC m=+2774.226559555" watchObservedRunningTime="2026-07-24 00:57:25.623239823 +0000 UTC m=+2774.228554139" Jul 24 00:57:25.941309 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:25.941204 2560 patch_prober.go:28] interesting pod/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 24 00:57:25.941309 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:25.941269 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" podUID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 24 00:57:26.941811 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:26.941777 2560 patch_prober.go:28] interesting pod/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 24 00:57:26.942204 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:26.941835 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" podUID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 24 00:57:27.540313 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:27.540273 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:27.945358 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:27.945272 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:27.945834 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:27.945669 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:27.946448 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:27.946429 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 00:57:37.609675 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:37.609570 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" Jul 24 00:57:37.610108 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:37.609897 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" Jul 24 00:57:38.624456 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:38.624422 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:57:45.026735 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:45.026702 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg_83dd244e-6328-49e0-862d-29492c8d1121/payload-processing/0.log" Jul 24 00:57:53.768862 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:57:53.768829 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:58:32.141611 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.141582 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg_83dd244e-6328-49e0-862d-29492c8d1121/payload-processing/0.log" Jul 24 00:58:32.418468 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.418365 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:58:32.720677 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.720584 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n"] Jul 24 00:58:32.725247 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.725213 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.728165 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.728141 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-78d793-openshift-default-dockercfg-hpjcp\"" Jul 24 00:58:32.737703 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.737673 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n"] Jul 24 00:58:32.807481 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.807442 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-workload-certs\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.807695 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.807537 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-istio-data\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.807695 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.807577 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12480620-2261-4624-b53b-d454b97cce3c-istio-token\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.807695 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.807601 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-istio-envoy\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.807695 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.807623 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nts8\" (UniqueName: \"kubernetes.io/projected/12480620-2261-4624-b53b-d454b97cce3c-kube-api-access-6nts8\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.807952 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.807714 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12480620-2261-4624-b53b-d454b97cce3c-istio-podinfo\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.807952 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.807757 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-workload-socket\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.807952 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.807825 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12480620-2261-4624-b53b-d454b97cce3c-istiod-ca-cert\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.807952 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.807910 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-credential-socket\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.908999 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.908958 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12480620-2261-4624-b53b-d454b97cce3c-istiod-ca-cert\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.909201 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.909034 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-credential-socket\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.909201 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.909084 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-workload-certs\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.909201 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.909100 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-istio-data\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.909201 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.909130 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12480620-2261-4624-b53b-d454b97cce3c-istio-token\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.909201 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.909156 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-istio-envoy\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.909201 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.909178 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nts8\" (UniqueName: \"kubernetes.io/projected/12480620-2261-4624-b53b-d454b97cce3c-kube-api-access-6nts8\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.909528 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.909210 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12480620-2261-4624-b53b-d454b97cce3c-istio-podinfo\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.909528 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.909251 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-workload-socket\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.909670 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.909529 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-istio-data\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.909670 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.909628 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-credential-socket\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.909840 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.909810 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-workload-certs\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.909928 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.909817 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-workload-socket\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.909928 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.909824 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12480620-2261-4624-b53b-d454b97cce3c-istiod-ca-cert\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.911526 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.911501 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-istio-envoy\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.911767 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.911746 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12480620-2261-4624-b53b-d454b97cce3c-istio-podinfo\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.917752 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.917728 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12480620-2261-4624-b53b-d454b97cce3c-istio-token\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:32.917956 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:32.917935 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nts8\" (UniqueName: \"kubernetes.io/projected/12480620-2261-4624-b53b-d454b97cce3c-kube-api-access-6nts8\") pod \"e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:33.037896 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:33.037865 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:33.183913 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:33.183886 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n"] Jul 24 00:58:33.185701 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:58:33.185671 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12480620_2261_4624_b53b_d454b97cce3c.slice/crio-5be3cc46959570076671cb254c48e53409eff2bc9e0ec383216ee4e98508ce5c WatchSource:0}: Error finding container 5be3cc46959570076671cb254c48e53409eff2bc9e0ec383216ee4e98508ce5c: Status 404 returned error can't find the container with id 5be3cc46959570076671cb254c48e53409eff2bc9e0ec383216ee4e98508ce5c Jul 24 00:58:33.187891 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:33.187860 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:58:33.187961 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:33.187929 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:58:33.187961 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:33.187955 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:58:33.701881 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:33.701829 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:58:33.954580 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:33.954479 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" event={"ID":"12480620-2261-4624-b53b-d454b97cce3c","Type":"ContainerStarted","Data":"770c6a35e797e4c217043c779dbf6567bbed204320ec36c1430c32f8fde05057"} Jul 24 00:58:33.954580 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:33.954543 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" event={"ID":"12480620-2261-4624-b53b-d454b97cce3c","Type":"ContainerStarted","Data":"5be3cc46959570076671cb254c48e53409eff2bc9e0ec383216ee4e98508ce5c"} Jul 24 00:58:33.979226 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:33.979171 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" podStartSLOduration=1.979152666 podStartE2EDuration="1.979152666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:58:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:58:33.976768103 +0000 UTC m=+2842.582082421" watchObservedRunningTime="2026-07-24 00:58:33.979152666 +0000 UTC m=+2842.584466982" Jul 24 00:58:34.038807 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:34.038775 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:34.043879 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:34.043847 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:34.959938 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:34.959899 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:34.961025 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:34.961006 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:35.949679 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:35.949620 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:58:36.034324 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.034284 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:58:36.095527 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.095491 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:58:36.521328 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.521293 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:58:36.621008 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.620962 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:58:36.699142 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.699100 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:58:36.704740 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.704708 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn"] Jul 24 00:58:36.708548 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.708526 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" Jul 24 00:58:36.711028 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.710996 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-78d793\"" Jul 24 00:58:36.711163 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.711057 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-78d793-dockercfg-2sccc\"" Jul 24 00:58:36.720282 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.720254 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn"] Jul 24 00:58:36.750472 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.750435 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2053d5b-c798-44d9-baf8-461bf1ea3b7a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn\" (UID: \"b2053d5b-c798-44d9-baf8-461bf1ea3b7a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" Jul 24 00:58:36.750679 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.750509 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6vpk\" (UniqueName: \"kubernetes.io/projected/b2053d5b-c798-44d9-baf8-461bf1ea3b7a-kube-api-access-v6vpk\") pod \"payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn\" (UID: \"b2053d5b-c798-44d9-baf8-461bf1ea3b7a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" Jul 24 00:58:36.851929 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.851819 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2053d5b-c798-44d9-baf8-461bf1ea3b7a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn\" (UID: \"b2053d5b-c798-44d9-baf8-461bf1ea3b7a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" Jul 24 00:58:36.851929 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.851909 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6vpk\" (UniqueName: \"kubernetes.io/projected/b2053d5b-c798-44d9-baf8-461bf1ea3b7a-kube-api-access-v6vpk\") pod \"payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn\" (UID: \"b2053d5b-c798-44d9-baf8-461bf1ea3b7a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" Jul 24 00:58:36.852755 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.852725 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2053d5b-c798-44d9-baf8-461bf1ea3b7a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn\" (UID: \"b2053d5b-c798-44d9-baf8-461bf1ea3b7a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" Jul 24 00:58:36.860628 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:36.860600 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6vpk\" (UniqueName: \"kubernetes.io/projected/b2053d5b-c798-44d9-baf8-461bf1ea3b7a-kube-api-access-v6vpk\") pod \"payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn\" (UID: \"b2053d5b-c798-44d9-baf8-461bf1ea3b7a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" Jul 24 00:58:37.021995 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:37.021955 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" Jul 24 00:58:37.162562 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:37.162527 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn"] Jul 24 00:58:37.164482 ip-10-0-139-82 kubenswrapper[2560]: W0724 00:58:37.164442 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2053d5b_c798_44d9_baf8_461bf1ea3b7a.slice/crio-a0f9adaf8e51fd82091884cb90594d00181018cb038dac5285ad93d36137b78d WatchSource:0}: Error finding container a0f9adaf8e51fd82091884cb90594d00181018cb038dac5285ad93d36137b78d: Status 404 returned error can't find the container with id a0f9adaf8e51fd82091884cb90594d00181018cb038dac5285ad93d36137b78d Jul 24 00:58:37.977089 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:37.977052 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" event={"ID":"b2053d5b-c798-44d9-baf8-461bf1ea3b7a","Type":"ContainerStarted","Data":"dcb39b68659a8d15415788405161adab697973f44531f9e88f1140975131f54c"} Jul 24 00:58:37.977287 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:37.977119 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" event={"ID":"b2053d5b-c798-44d9-baf8-461bf1ea3b7a","Type":"ContainerStarted","Data":"a0f9adaf8e51fd82091884cb90594d00181018cb038dac5285ad93d36137b78d"} Jul 24 00:58:37.977287 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:37.977150 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" Jul 24 00:58:37.997137 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:37.997080 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" podStartSLOduration=1.997063142 podStartE2EDuration="1.997063142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 00:58:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 00:58:37.995948705 +0000 UTC m=+2846.601263033" watchObservedRunningTime="2026-07-24 00:58:37.997063142 +0000 UTC m=+2846.602377457" Jul 24 00:58:40.475691 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:40.475624 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:58:48.985298 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:48.985261 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" Jul 24 00:58:49.961779 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:49.961738 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 24 00:58:49.962011 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:49.961797 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" podUID="12480620-2261-4624-b53b-d454b97cce3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 24 00:58:50.046089 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:50.046058 2560 generic.go:356] "Generic (PLEG): container finished" podID="12480620-2261-4624-b53b-d454b97cce3c" containerID="770c6a35e797e4c217043c779dbf6567bbed204320ec36c1430c32f8fde05057" exitCode=0 Jul 24 00:58:50.046480 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:50.046130 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" event={"ID":"12480620-2261-4624-b53b-d454b97cce3c","Type":"ContainerDied","Data":"770c6a35e797e4c217043c779dbf6567bbed204320ec36c1430c32f8fde05057"} Jul 24 00:58:50.046571 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:50.046548 2560 scope.go:117] "RemoveContainer" containerID="770c6a35e797e4c217043c779dbf6567bbed204320ec36c1430c32f8fde05057" Jul 24 00:58:50.047670 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:50.047621 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:58:50.047759 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:50.047705 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:58:50.047759 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:50.047749 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 00:58:51.053780 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:51.053747 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" event={"ID":"12480620-2261-4624-b53b-d454b97cce3c","Type":"ContainerStarted","Data":"ba4d8895985c304c87c80537b062f0c83ff89ba2d3376b6283f3b526601bc7f6"} Jul 24 00:58:52.038718 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:52.038687 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:52.039001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:52.038878 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 24 00:58:52.039001 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:52.038922 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" podUID="12480620-2261-4624-b53b-d454b97cce3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 24 00:58:53.038577 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:53.038542 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 24 00:58:53.038997 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:53.038598 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" podUID="12480620-2261-4624-b53b-d454b97cce3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 24 00:58:54.039227 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:54.039184 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 24 00:58:54.039668 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:54.039245 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" podUID="12480620-2261-4624-b53b-d454b97cce3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 24 00:58:55.042966 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:55.042931 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:55.043357 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:55.043314 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:58:55.044112 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:58:55.044094 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:59:01.996435 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:01.996396 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:59:02.029556 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:02.029521 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn"] Jul 24 00:59:02.030081 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:02.030024 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" podUID="b2053d5b-c798-44d9-baf8-461bf1ea3b7a" containerName="payload-pre-processing" containerID="cri-o://dcb39b68659a8d15415788405161adab697973f44531f9e88f1140975131f54c" gracePeriod=30 Jul 24 00:59:02.061745 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:02.061714 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:59:02.122142 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:02.122110 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:59:02.324823 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:02.324794 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" Jul 24 00:59:02.414973 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:02.414938 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6vpk\" (UniqueName: \"kubernetes.io/projected/b2053d5b-c798-44d9-baf8-461bf1ea3b7a-kube-api-access-v6vpk\") pod \"b2053d5b-c798-44d9-baf8-461bf1ea3b7a\" (UID: \"b2053d5b-c798-44d9-baf8-461bf1ea3b7a\") " Jul 24 00:59:02.415179 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:02.415038 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2053d5b-c798-44d9-baf8-461bf1ea3b7a-plugins-config-volume\") pod \"b2053d5b-c798-44d9-baf8-461bf1ea3b7a\" (UID: \"b2053d5b-c798-44d9-baf8-461bf1ea3b7a\") " Jul 24 00:59:02.415409 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:02.415387 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2053d5b-c798-44d9-baf8-461bf1ea3b7a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2053d5b-c798-44d9-baf8-461bf1ea3b7a" (UID: "b2053d5b-c798-44d9-baf8-461bf1ea3b7a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:59:02.417219 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:02.417194 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2053d5b-c798-44d9-baf8-461bf1ea3b7a-kube-api-access-v6vpk" (OuterVolumeSpecName: "kube-api-access-v6vpk") pod "b2053d5b-c798-44d9-baf8-461bf1ea3b7a" (UID: "b2053d5b-c798-44d9-baf8-461bf1ea3b7a"). InnerVolumeSpecName "kube-api-access-v6vpk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:59:02.516631 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:02.516593 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6vpk\" (UniqueName: \"kubernetes.io/projected/b2053d5b-c798-44d9-baf8-461bf1ea3b7a-kube-api-access-v6vpk\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:59:02.516631 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:02.516634 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2053d5b-c798-44d9-baf8-461bf1ea3b7a-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:59:03.122541 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:03.122457 2560 generic.go:356] "Generic (PLEG): container finished" podID="b2053d5b-c798-44d9-baf8-461bf1ea3b7a" containerID="dcb39b68659a8d15415788405161adab697973f44531f9e88f1140975131f54c" exitCode=0 Jul 24 00:59:03.122541 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:03.122505 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" Jul 24 00:59:03.123024 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:03.122535 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" event={"ID":"b2053d5b-c798-44d9-baf8-461bf1ea3b7a","Type":"ContainerDied","Data":"dcb39b68659a8d15415788405161adab697973f44531f9e88f1140975131f54c"} Jul 24 00:59:03.123024 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:03.122614 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn" event={"ID":"b2053d5b-c798-44d9-baf8-461bf1ea3b7a","Type":"ContainerDied","Data":"a0f9adaf8e51fd82091884cb90594d00181018cb038dac5285ad93d36137b78d"} Jul 24 00:59:03.123024 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:03.122635 2560 scope.go:117] "RemoveContainer" containerID="dcb39b68659a8d15415788405161adab697973f44531f9e88f1140975131f54c" Jul 24 00:59:03.132426 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:03.132404 2560 scope.go:117] "RemoveContainer" containerID="dcb39b68659a8d15415788405161adab697973f44531f9e88f1140975131f54c" Jul 24 00:59:03.132727 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:59:03.132707 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcb39b68659a8d15415788405161adab697973f44531f9e88f1140975131f54c\": container with ID starting with dcb39b68659a8d15415788405161adab697973f44531f9e88f1140975131f54c not found: ID does not exist" containerID="dcb39b68659a8d15415788405161adab697973f44531f9e88f1140975131f54c" Jul 24 00:59:03.132800 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:03.132737 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcb39b68659a8d15415788405161adab697973f44531f9e88f1140975131f54c"} err="failed to get container status \"dcb39b68659a8d15415788405161adab697973f44531f9e88f1140975131f54c\": rpc error: code = NotFound desc = could not find container \"dcb39b68659a8d15415788405161adab697973f44531f9e88f1140975131f54c\": container with ID starting with dcb39b68659a8d15415788405161adab697973f44531f9e88f1140975131f54c not found: ID does not exist" Jul 24 00:59:03.145238 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:03.145207 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn"] Jul 24 00:59:03.149012 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:03.148983 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78d793-58c455bcb4-7mbsn"] Jul 24 00:59:04.349013 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:04.348979 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2053d5b-c798-44d9-baf8-461bf1ea3b7a" path="/var/lib/kubelet/pods/b2053d5b-c798-44d9-baf8-461bf1ea3b7a/volumes" Jul 24 00:59:06.501132 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:06.501098 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:59:06.901378 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:06.901294 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:59:12.461128 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:12.461091 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:59:12.517361 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:12.517321 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:59:12.763874 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:12.763840 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n"] Jul 24 00:59:12.764103 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:12.764065 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" podUID="12480620-2261-4624-b53b-d454b97cce3c" containerName="istio-proxy" containerID="cri-o://ba4d8895985c304c87c80537b062f0c83ff89ba2d3376b6283f3b526601bc7f6" gracePeriod=30 Jul 24 00:59:12.810527 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:12.810495 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:59:12.980727 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:12.980689 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:59:18.033272 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.033197 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:59:18.077512 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.077474 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12480620-2261-4624-b53b-d454b97cce3c-istio-podinfo\") pod \"12480620-2261-4624-b53b-d454b97cce3c\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " Jul 24 00:59:18.077766 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.077536 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-credential-socket\") pod \"12480620-2261-4624-b53b-d454b97cce3c\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " Jul 24 00:59:18.077766 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.077575 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12480620-2261-4624-b53b-d454b97cce3c-istiod-ca-cert\") pod \"12480620-2261-4624-b53b-d454b97cce3c\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " Jul 24 00:59:18.077766 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.077620 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nts8\" (UniqueName: \"kubernetes.io/projected/12480620-2261-4624-b53b-d454b97cce3c-kube-api-access-6nts8\") pod \"12480620-2261-4624-b53b-d454b97cce3c\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " Jul 24 00:59:18.077766 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.077713 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-workload-socket\") pod \"12480620-2261-4624-b53b-d454b97cce3c\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " Jul 24 00:59:18.078004 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.077973 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "12480620-2261-4624-b53b-d454b97cce3c" (UID: "12480620-2261-4624-b53b-d454b97cce3c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:59:18.078067 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.078012 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "12480620-2261-4624-b53b-d454b97cce3c" (UID: "12480620-2261-4624-b53b-d454b97cce3c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:59:18.078067 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.078032 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12480620-2261-4624-b53b-d454b97cce3c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "12480620-2261-4624-b53b-d454b97cce3c" (UID: "12480620-2261-4624-b53b-d454b97cce3c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 00:59:18.078181 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.078121 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-istio-data\") pod \"12480620-2261-4624-b53b-d454b97cce3c\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " Jul 24 00:59:18.078236 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.078219 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12480620-2261-4624-b53b-d454b97cce3c-istio-token\") pod \"12480620-2261-4624-b53b-d454b97cce3c\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " Jul 24 00:59:18.078291 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.078246 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-istio-envoy\") pod \"12480620-2261-4624-b53b-d454b97cce3c\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " Jul 24 00:59:18.078291 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.078279 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-workload-certs\") pod \"12480620-2261-4624-b53b-d454b97cce3c\" (UID: \"12480620-2261-4624-b53b-d454b97cce3c\") " Jul 24 00:59:18.078921 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.078672 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-credential-socket\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:59:18.078921 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.078697 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12480620-2261-4624-b53b-d454b97cce3c-istiod-ca-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:59:18.078921 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.078712 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-workload-socket\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:59:18.078921 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.078885 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "12480620-2261-4624-b53b-d454b97cce3c" (UID: "12480620-2261-4624-b53b-d454b97cce3c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:59:18.079224 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.079135 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-istio-data" (OuterVolumeSpecName: "istio-data") pod "12480620-2261-4624-b53b-d454b97cce3c" (UID: "12480620-2261-4624-b53b-d454b97cce3c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:59:18.080075 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.080052 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12480620-2261-4624-b53b-d454b97cce3c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "12480620-2261-4624-b53b-d454b97cce3c" (UID: "12480620-2261-4624-b53b-d454b97cce3c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 00:59:18.080211 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.080122 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12480620-2261-4624-b53b-d454b97cce3c-kube-api-access-6nts8" (OuterVolumeSpecName: "kube-api-access-6nts8") pod "12480620-2261-4624-b53b-d454b97cce3c" (UID: "12480620-2261-4624-b53b-d454b97cce3c"). InnerVolumeSpecName "kube-api-access-6nts8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:59:18.080436 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.080421 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12480620-2261-4624-b53b-d454b97cce3c-istio-token" (OuterVolumeSpecName: "istio-token") pod "12480620-2261-4624-b53b-d454b97cce3c" (UID: "12480620-2261-4624-b53b-d454b97cce3c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 00:59:18.080959 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.080944 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "12480620-2261-4624-b53b-d454b97cce3c" (UID: "12480620-2261-4624-b53b-d454b97cce3c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 00:59:18.179488 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.179447 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12480620-2261-4624-b53b-d454b97cce3c-istio-podinfo\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:59:18.179488 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.179481 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nts8\" (UniqueName: \"kubernetes.io/projected/12480620-2261-4624-b53b-d454b97cce3c-kube-api-access-6nts8\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:59:18.179488 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.179493 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-istio-data\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:59:18.179775 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.179502 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12480620-2261-4624-b53b-d454b97cce3c-istio-token\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:59:18.179775 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.179510 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-istio-envoy\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:59:18.179775 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.179517 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12480620-2261-4624-b53b-d454b97cce3c-workload-certs\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 00:59:18.206490 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.206462 2560 generic.go:356] "Generic (PLEG): container finished" podID="12480620-2261-4624-b53b-d454b97cce3c" containerID="ba4d8895985c304c87c80537b062f0c83ff89ba2d3376b6283f3b526601bc7f6" exitCode=0 Jul 24 00:59:18.206666 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.206517 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" Jul 24 00:59:18.206666 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.206551 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" event={"ID":"12480620-2261-4624-b53b-d454b97cce3c","Type":"ContainerDied","Data":"ba4d8895985c304c87c80537b062f0c83ff89ba2d3376b6283f3b526601bc7f6"} Jul 24 00:59:18.206666 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.206608 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n" event={"ID":"12480620-2261-4624-b53b-d454b97cce3c","Type":"ContainerDied","Data":"5be3cc46959570076671cb254c48e53409eff2bc9e0ec383216ee4e98508ce5c"} Jul 24 00:59:18.206666 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.206632 2560 scope.go:117] "RemoveContainer" containerID="ba4d8895985c304c87c80537b062f0c83ff89ba2d3376b6283f3b526601bc7f6" Jul 24 00:59:18.216843 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.216825 2560 scope.go:117] "RemoveContainer" containerID="770c6a35e797e4c217043c779dbf6567bbed204320ec36c1430c32f8fde05057" Jul 24 00:59:18.225231 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.225213 2560 scope.go:117] "RemoveContainer" containerID="ba4d8895985c304c87c80537b062f0c83ff89ba2d3376b6283f3b526601bc7f6" Jul 24 00:59:18.225548 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:59:18.225524 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba4d8895985c304c87c80537b062f0c83ff89ba2d3376b6283f3b526601bc7f6\": container with ID starting with ba4d8895985c304c87c80537b062f0c83ff89ba2d3376b6283f3b526601bc7f6 not found: ID does not exist" containerID="ba4d8895985c304c87c80537b062f0c83ff89ba2d3376b6283f3b526601bc7f6" Jul 24 00:59:18.225627 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.225559 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba4d8895985c304c87c80537b062f0c83ff89ba2d3376b6283f3b526601bc7f6"} err="failed to get container status \"ba4d8895985c304c87c80537b062f0c83ff89ba2d3376b6283f3b526601bc7f6\": rpc error: code = NotFound desc = could not find container \"ba4d8895985c304c87c80537b062f0c83ff89ba2d3376b6283f3b526601bc7f6\": container with ID starting with ba4d8895985c304c87c80537b062f0c83ff89ba2d3376b6283f3b526601bc7f6 not found: ID does not exist" Jul 24 00:59:18.225627 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.225577 2560 scope.go:117] "RemoveContainer" containerID="770c6a35e797e4c217043c779dbf6567bbed204320ec36c1430c32f8fde05057" Jul 24 00:59:18.226240 ip-10-0-139-82 kubenswrapper[2560]: E0724 00:59:18.226153 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"770c6a35e797e4c217043c779dbf6567bbed204320ec36c1430c32f8fde05057\": container with ID starting with 770c6a35e797e4c217043c779dbf6567bbed204320ec36c1430c32f8fde05057 not found: ID does not exist" containerID="770c6a35e797e4c217043c779dbf6567bbed204320ec36c1430c32f8fde05057" Jul 24 00:59:18.226240 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.226214 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"770c6a35e797e4c217043c779dbf6567bbed204320ec36c1430c32f8fde05057"} err="failed to get container status \"770c6a35e797e4c217043c779dbf6567bbed204320ec36c1430c32f8fde05057\": rpc error: code = NotFound desc = could not find container \"770c6a35e797e4c217043c779dbf6567bbed204320ec36c1430c32f8fde05057\": container with ID starting with 770c6a35e797e4c217043c779dbf6567bbed204320ec36c1430c32f8fde05057 not found: ID does not exist" Jul 24 00:59:18.228801 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.228781 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n"] Jul 24 00:59:18.232290 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.232262 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-78d793-openshift-default-7689ff9688-ct84n"] Jul 24 00:59:18.354324 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:18.350743 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12480620-2261-4624-b53b-d454b97cce3c" path="/var/lib/kubelet/pods/12480620-2261-4624-b53b-d454b97cce3c/volumes" Jul 24 00:59:32.206744 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:32.206711 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:59:32.272772 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:32.272731 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:59:32.334176 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:32.334116 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:59:36.668151 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:36.668109 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:59:37.099207 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:37.099172 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 00:59:42.181958 ip-10-0-139-82 kubenswrapper[2560]: I0724 00:59:42.181922 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:18.360914 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:18.360868 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:19.144284 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:19.144246 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:24.695379 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:24.695343 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:24.751965 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:24.751932 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:24.966227 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:24.966122 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt"] Jul 24 01:00:24.966404 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:24.966349 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" podUID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerName="istio-proxy" containerID="cri-o://45321e6485dddb1a7561dfb42c535c6875f688fb201deb9e14191b3f620f82a4" gracePeriod=30 Jul 24 01:00:24.987837 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:24.987793 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:25.216856 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:25.216764 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:27.948354 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:27.948313 2560 patch_prober.go:28] interesting pod/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 01:00:27.948757 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:27.948387 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" podUID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 01:00:30.232622 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.232559 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 01:00:30.324181 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.324146 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-data\") pod \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " Jul 24 01:00:30.324181 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.324183 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-credential-socket\") pod \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " Jul 24 01:00:30.324422 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.324213 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-workload-certs\") pod \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " Jul 24 01:00:30.324422 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.324242 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-token\") pod \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " Jul 24 01:00:30.324422 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.324258 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvvcw\" (UniqueName: \"kubernetes.io/projected/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-kube-api-access-nvvcw\") pod \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " Jul 24 01:00:30.324422 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.324310 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-workload-socket\") pod \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " Jul 24 01:00:30.324422 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.324366 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-envoy\") pod \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " Jul 24 01:00:30.324731 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.324434 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istiod-ca-cert\") pod \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " Jul 24 01:00:30.324731 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.324457 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "edf2518a-9c33-4160-ad0f-ffbafc54c2d9" (UID: "edf2518a-9c33-4160-ad0f-ffbafc54c2d9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 01:00:30.324731 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.324473 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-podinfo\") pod \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\" (UID: \"edf2518a-9c33-4160-ad0f-ffbafc54c2d9\") " Jul 24 01:00:30.324731 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.324702 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "edf2518a-9c33-4160-ad0f-ffbafc54c2d9" (UID: "edf2518a-9c33-4160-ad0f-ffbafc54c2d9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 01:00:30.324940 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.324721 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "edf2518a-9c33-4160-ad0f-ffbafc54c2d9" (UID: "edf2518a-9c33-4160-ad0f-ffbafc54c2d9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 01:00:30.325024 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.324984 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "edf2518a-9c33-4160-ad0f-ffbafc54c2d9" (UID: "edf2518a-9c33-4160-ad0f-ffbafc54c2d9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 01:00:30.325024 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.325001 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-credential-socket\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:30.325024 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.325023 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-workload-certs\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:30.325272 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.325037 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-workload-socket\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:30.325272 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.325179 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-data" (OuterVolumeSpecName: "istio-data") pod "edf2518a-9c33-4160-ad0f-ffbafc54c2d9" (UID: "edf2518a-9c33-4160-ad0f-ffbafc54c2d9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 01:00:30.326559 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.326538 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "edf2518a-9c33-4160-ad0f-ffbafc54c2d9" (UID: "edf2518a-9c33-4160-ad0f-ffbafc54c2d9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 01:00:30.326796 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.326701 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "edf2518a-9c33-4160-ad0f-ffbafc54c2d9" (UID: "edf2518a-9c33-4160-ad0f-ffbafc54c2d9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 01:00:30.326863 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.326807 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-kube-api-access-nvvcw" (OuterVolumeSpecName: "kube-api-access-nvvcw") pod "edf2518a-9c33-4160-ad0f-ffbafc54c2d9" (UID: "edf2518a-9c33-4160-ad0f-ffbafc54c2d9"). InnerVolumeSpecName "kube-api-access-nvvcw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:00:30.326863 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.326850 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-token" (OuterVolumeSpecName: "istio-token") pod "edf2518a-9c33-4160-ad0f-ffbafc54c2d9" (UID: "edf2518a-9c33-4160-ad0f-ffbafc54c2d9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:00:30.425959 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.425920 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istiod-ca-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:30.425959 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.425956 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-podinfo\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:30.425959 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.425965 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-data\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:30.425959 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.425973 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-token\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:30.426252 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.425981 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvvcw\" (UniqueName: \"kubernetes.io/projected/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-kube-api-access-nvvcw\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:30.426252 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.425991 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edf2518a-9c33-4160-ad0f-ffbafc54c2d9-istio-envoy\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:30.603106 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.603072 2560 generic.go:356] "Generic (PLEG): container finished" podID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerID="45321e6485dddb1a7561dfb42c535c6875f688fb201deb9e14191b3f620f82a4" exitCode=0 Jul 24 01:00:30.603294 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.603122 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" event={"ID":"edf2518a-9c33-4160-ad0f-ffbafc54c2d9","Type":"ContainerDied","Data":"45321e6485dddb1a7561dfb42c535c6875f688fb201deb9e14191b3f620f82a4"} Jul 24 01:00:30.603294 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.603134 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" Jul 24 01:00:30.603294 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.603190 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt" event={"ID":"edf2518a-9c33-4160-ad0f-ffbafc54c2d9","Type":"ContainerDied","Data":"fc492d81415f15d4dce8a849aa7da3897cd224b4aa5915be8f064dbcdf2c7fb4"} Jul 24 01:00:30.603294 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.603213 2560 scope.go:117] "RemoveContainer" containerID="45321e6485dddb1a7561dfb42c535c6875f688fb201deb9e14191b3f620f82a4" Jul 24 01:00:30.613860 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.613840 2560 scope.go:117] "RemoveContainer" containerID="9a4cdb1bd17594400ed07d0952e8a02d6694a36b2f6d504bbf9d0d66559ca0b7" Jul 24 01:00:30.620684 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.620660 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt"] Jul 24 01:00:30.623149 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.623128 2560 scope.go:117] "RemoveContainer" containerID="45321e6485dddb1a7561dfb42c535c6875f688fb201deb9e14191b3f620f82a4" Jul 24 01:00:30.623440 ip-10-0-139-82 kubenswrapper[2560]: E0724 01:00:30.623422 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45321e6485dddb1a7561dfb42c535c6875f688fb201deb9e14191b3f620f82a4\": container with ID starting with 45321e6485dddb1a7561dfb42c535c6875f688fb201deb9e14191b3f620f82a4 not found: ID does not exist" containerID="45321e6485dddb1a7561dfb42c535c6875f688fb201deb9e14191b3f620f82a4" Jul 24 01:00:30.623499 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.623447 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45321e6485dddb1a7561dfb42c535c6875f688fb201deb9e14191b3f620f82a4"} err="failed to get container status \"45321e6485dddb1a7561dfb42c535c6875f688fb201deb9e14191b3f620f82a4\": rpc error: code = NotFound desc = could not find container \"45321e6485dddb1a7561dfb42c535c6875f688fb201deb9e14191b3f620f82a4\": container with ID starting with 45321e6485dddb1a7561dfb42c535c6875f688fb201deb9e14191b3f620f82a4 not found: ID does not exist" Jul 24 01:00:30.623499 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.623464 2560 scope.go:117] "RemoveContainer" containerID="9a4cdb1bd17594400ed07d0952e8a02d6694a36b2f6d504bbf9d0d66559ca0b7" Jul 24 01:00:30.623706 ip-10-0-139-82 kubenswrapper[2560]: E0724 01:00:30.623688 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a4cdb1bd17594400ed07d0952e8a02d6694a36b2f6d504bbf9d0d66559ca0b7\": container with ID starting with 9a4cdb1bd17594400ed07d0952e8a02d6694a36b2f6d504bbf9d0d66559ca0b7 not found: ID does not exist" containerID="9a4cdb1bd17594400ed07d0952e8a02d6694a36b2f6d504bbf9d0d66559ca0b7" Jul 24 01:00:30.623760 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.623711 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a4cdb1bd17594400ed07d0952e8a02d6694a36b2f6d504bbf9d0d66559ca0b7"} err="failed to get container status \"9a4cdb1bd17594400ed07d0952e8a02d6694a36b2f6d504bbf9d0d66559ca0b7\": rpc error: code = NotFound desc = could not find container \"9a4cdb1bd17594400ed07d0952e8a02d6694a36b2f6d504bbf9d0d66559ca0b7\": container with ID starting with 9a4cdb1bd17594400ed07d0952e8a02d6694a36b2f6d504bbf9d0d66559ca0b7 not found: ID does not exist" Jul 24 01:00:30.624107 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:30.624089 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-193ff5-openshift-default-7df9bf969d-k2kmt"] Jul 24 01:00:32.348812 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:32.348777 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" path="/var/lib/kubelet/pods/edf2518a-9c33-4160-ad0f-ffbafc54c2d9/volumes" Jul 24 01:00:35.903363 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:35.903285 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:35.982482 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:35.982447 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg"] Jul 24 01:00:35.982713 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:35.982686 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" podUID="83dd244e-6328-49e0-862d-29492c8d1121" containerName="payload-processing" containerID="cri-o://3447ad77e56c0d00fc98deb3c559f8d2d00920d7eb16647d2a60553ed3b13af4" gracePeriod=30 Jul 24 01:00:35.993327 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:35.993191 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x"] Jul 24 01:00:35.993527 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:35.993478 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" podUID="323ab7ae-1d94-42cc-bf67-914ba3033fde" containerName="payload-pre-processing" containerID="cri-o://668978be11be9834ad7359e7b5b205c2c20e5690437c5593804b57a7e3e35a70" gracePeriod=30 Jul 24 01:00:36.041100 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.041061 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:36.136211 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.136168 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:36.254575 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.254549 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" Jul 24 01:00:36.284376 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.284353 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" Jul 24 01:00:36.387535 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.387497 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27nh7\" (UniqueName: \"kubernetes.io/projected/323ab7ae-1d94-42cc-bf67-914ba3033fde-kube-api-access-27nh7\") pod \"323ab7ae-1d94-42cc-bf67-914ba3033fde\" (UID: \"323ab7ae-1d94-42cc-bf67-914ba3033fde\") " Jul 24 01:00:36.387738 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.387587 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/323ab7ae-1d94-42cc-bf67-914ba3033fde-plugins-config-volume\") pod \"323ab7ae-1d94-42cc-bf67-914ba3033fde\" (UID: \"323ab7ae-1d94-42cc-bf67-914ba3033fde\") " Jul 24 01:00:36.387738 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.387620 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpzsb\" (UniqueName: \"kubernetes.io/projected/83dd244e-6328-49e0-862d-29492c8d1121-kube-api-access-qpzsb\") pod \"83dd244e-6328-49e0-862d-29492c8d1121\" (UID: \"83dd244e-6328-49e0-862d-29492c8d1121\") " Jul 24 01:00:36.387738 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.387686 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83dd244e-6328-49e0-862d-29492c8d1121-plugins-config-volume\") pod \"83dd244e-6328-49e0-862d-29492c8d1121\" (UID: \"83dd244e-6328-49e0-862d-29492c8d1121\") " Jul 24 01:00:36.388167 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.388137 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83dd244e-6328-49e0-862d-29492c8d1121-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "83dd244e-6328-49e0-862d-29492c8d1121" (UID: "83dd244e-6328-49e0-862d-29492c8d1121"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 01:00:36.388294 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.388146 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/323ab7ae-1d94-42cc-bf67-914ba3033fde-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "323ab7ae-1d94-42cc-bf67-914ba3033fde" (UID: "323ab7ae-1d94-42cc-bf67-914ba3033fde"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 01:00:36.389748 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.389723 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83dd244e-6328-49e0-862d-29492c8d1121-kube-api-access-qpzsb" (OuterVolumeSpecName: "kube-api-access-qpzsb") pod "83dd244e-6328-49e0-862d-29492c8d1121" (UID: "83dd244e-6328-49e0-862d-29492c8d1121"). InnerVolumeSpecName "kube-api-access-qpzsb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:00:36.389853 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.389783 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/323ab7ae-1d94-42cc-bf67-914ba3033fde-kube-api-access-27nh7" (OuterVolumeSpecName: "kube-api-access-27nh7") pod "323ab7ae-1d94-42cc-bf67-914ba3033fde" (UID: "323ab7ae-1d94-42cc-bf67-914ba3033fde"). InnerVolumeSpecName "kube-api-access-27nh7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:00:36.488907 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.488814 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/323ab7ae-1d94-42cc-bf67-914ba3033fde-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:36.488907 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.488850 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpzsb\" (UniqueName: \"kubernetes.io/projected/83dd244e-6328-49e0-862d-29492c8d1121-kube-api-access-qpzsb\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:36.488907 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.488859 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83dd244e-6328-49e0-862d-29492c8d1121-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:36.488907 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.488868 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27nh7\" (UniqueName: \"kubernetes.io/projected/323ab7ae-1d94-42cc-bf67-914ba3033fde-kube-api-access-27nh7\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:36.636504 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.636463 2560 generic.go:356] "Generic (PLEG): container finished" podID="83dd244e-6328-49e0-862d-29492c8d1121" containerID="3447ad77e56c0d00fc98deb3c559f8d2d00920d7eb16647d2a60553ed3b13af4" exitCode=0 Jul 24 01:00:36.636736 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.636532 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" Jul 24 01:00:36.636736 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.636548 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" event={"ID":"83dd244e-6328-49e0-862d-29492c8d1121","Type":"ContainerDied","Data":"3447ad77e56c0d00fc98deb3c559f8d2d00920d7eb16647d2a60553ed3b13af4"} Jul 24 01:00:36.636736 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.636669 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg" event={"ID":"83dd244e-6328-49e0-862d-29492c8d1121","Type":"ContainerDied","Data":"2b99d217668d01bbae66c5ccb5779579ce54ba97ff0908b23e5591f14b2ff501"} Jul 24 01:00:36.636736 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.636703 2560 scope.go:117] "RemoveContainer" containerID="3447ad77e56c0d00fc98deb3c559f8d2d00920d7eb16647d2a60553ed3b13af4" Jul 24 01:00:36.638039 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.638016 2560 generic.go:356] "Generic (PLEG): container finished" podID="323ab7ae-1d94-42cc-bf67-914ba3033fde" containerID="668978be11be9834ad7359e7b5b205c2c20e5690437c5593804b57a7e3e35a70" exitCode=0 Jul 24 01:00:36.638145 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.638048 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" event={"ID":"323ab7ae-1d94-42cc-bf67-914ba3033fde","Type":"ContainerDied","Data":"668978be11be9834ad7359e7b5b205c2c20e5690437c5593804b57a7e3e35a70"} Jul 24 01:00:36.638145 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.638070 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" Jul 24 01:00:36.638145 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.638088 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x" event={"ID":"323ab7ae-1d94-42cc-bf67-914ba3033fde","Type":"ContainerDied","Data":"1ff09ca547f9839cb457bdc0b3dbd8c62c916e0f1f7b4efa5009503764797665"} Jul 24 01:00:36.647504 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.647486 2560 scope.go:117] "RemoveContainer" containerID="3447ad77e56c0d00fc98deb3c559f8d2d00920d7eb16647d2a60553ed3b13af4" Jul 24 01:00:36.647864 ip-10-0-139-82 kubenswrapper[2560]: E0724 01:00:36.647834 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3447ad77e56c0d00fc98deb3c559f8d2d00920d7eb16647d2a60553ed3b13af4\": container with ID starting with 3447ad77e56c0d00fc98deb3c559f8d2d00920d7eb16647d2a60553ed3b13af4 not found: ID does not exist" containerID="3447ad77e56c0d00fc98deb3c559f8d2d00920d7eb16647d2a60553ed3b13af4" Jul 24 01:00:36.647937 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.647877 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3447ad77e56c0d00fc98deb3c559f8d2d00920d7eb16647d2a60553ed3b13af4"} err="failed to get container status \"3447ad77e56c0d00fc98deb3c559f8d2d00920d7eb16647d2a60553ed3b13af4\": rpc error: code = NotFound desc = could not find container \"3447ad77e56c0d00fc98deb3c559f8d2d00920d7eb16647d2a60553ed3b13af4\": container with ID starting with 3447ad77e56c0d00fc98deb3c559f8d2d00920d7eb16647d2a60553ed3b13af4 not found: ID does not exist" Jul 24 01:00:36.648001 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.647935 2560 scope.go:117] "RemoveContainer" containerID="668978be11be9834ad7359e7b5b205c2c20e5690437c5593804b57a7e3e35a70" Jul 24 01:00:36.655998 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.655978 2560 scope.go:117] "RemoveContainer" containerID="668978be11be9834ad7359e7b5b205c2c20e5690437c5593804b57a7e3e35a70" Jul 24 01:00:36.656276 ip-10-0-139-82 kubenswrapper[2560]: E0724 01:00:36.656254 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"668978be11be9834ad7359e7b5b205c2c20e5690437c5593804b57a7e3e35a70\": container with ID starting with 668978be11be9834ad7359e7b5b205c2c20e5690437c5593804b57a7e3e35a70 not found: ID does not exist" containerID="668978be11be9834ad7359e7b5b205c2c20e5690437c5593804b57a7e3e35a70" Jul 24 01:00:36.656339 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.656282 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"668978be11be9834ad7359e7b5b205c2c20e5690437c5593804b57a7e3e35a70"} err="failed to get container status \"668978be11be9834ad7359e7b5b205c2c20e5690437c5593804b57a7e3e35a70\": rpc error: code = NotFound desc = could not find container \"668978be11be9834ad7359e7b5b205c2c20e5690437c5593804b57a7e3e35a70\": container with ID starting with 668978be11be9834ad7359e7b5b205c2c20e5690437c5593804b57a7e3e35a70 not found: ID does not exist" Jul 24 01:00:36.660216 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.660196 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg"] Jul 24 01:00:36.662065 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.662041 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4fa1d5-69ff6cc98-cm4zg"] Jul 24 01:00:36.672016 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.671985 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x"] Jul 24 01:00:36.675550 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:36.675528 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-4fa1d5-6f4b947f68-rb87x"] Jul 24 01:00:38.348460 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:38.348432 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="323ab7ae-1d94-42cc-bf67-914ba3033fde" path="/var/lib/kubelet/pods/323ab7ae-1d94-42cc-bf67-914ba3033fde/volumes" Jul 24 01:00:38.348877 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:38.348794 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83dd244e-6328-49e0-862d-29492c8d1121" path="/var/lib/kubelet/pods/83dd244e-6328-49e0-862d-29492c8d1121/volumes" Jul 24 01:00:40.417439 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:40.417403 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:40.815526 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:40.815490 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:46.857503 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:46.857470 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:46.953774 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:46.953739 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:47.135728 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:47.135632 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:47.374865 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:47.374833 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:58.065101 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.065064 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:58.099028 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.098195 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn"] Jul 24 01:00:58.099028 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.098813 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" podUID="cd832f51-24ab-4f24-89cc-1332a4233318" containerName="payload-processing" containerID="cri-o://7b003074d2d34b6fbe146a6fe599002eaacefae79dbe8d3d2054c615c0105273" gracePeriod=30 Jul 24 01:00:58.102801 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.102771 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v"] Jul 24 01:00:58.103218 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.103169 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" podUID="dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8" containerName="payload-pre-processing" containerID="cri-o://46efae517e36d931c08c037ff80aa920b3975b68701754dde71d0c58f1110acc" gracePeriod=30 Jul 24 01:00:58.139203 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.139174 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:58.199285 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.199248 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:00:58.363984 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.363951 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" Jul 24 01:00:58.393916 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.393891 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" Jul 24 01:00:58.410265 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.410229 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfzz7\" (UniqueName: \"kubernetes.io/projected/cd832f51-24ab-4f24-89cc-1332a4233318-kube-api-access-qfzz7\") pod \"cd832f51-24ab-4f24-89cc-1332a4233318\" (UID: \"cd832f51-24ab-4f24-89cc-1332a4233318\") " Jul 24 01:00:58.410489 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.410329 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd832f51-24ab-4f24-89cc-1332a4233318-plugins-config-volume\") pod \"cd832f51-24ab-4f24-89cc-1332a4233318\" (UID: \"cd832f51-24ab-4f24-89cc-1332a4233318\") " Jul 24 01:00:58.410489 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.410391 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8-plugins-config-volume\") pod \"dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8\" (UID: \"dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8\") " Jul 24 01:00:58.410489 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.410422 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2j7ff\" (UniqueName: \"kubernetes.io/projected/dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8-kube-api-access-2j7ff\") pod \"dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8\" (UID: \"dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8\") " Jul 24 01:00:58.411146 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.411103 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8" (UID: "dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 01:00:58.411494 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.411450 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd832f51-24ab-4f24-89cc-1332a4233318-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cd832f51-24ab-4f24-89cc-1332a4233318" (UID: "cd832f51-24ab-4f24-89cc-1332a4233318"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 01:00:58.412371 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.412344 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd832f51-24ab-4f24-89cc-1332a4233318-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:58.412371 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.412377 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:58.413316 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.413290 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd832f51-24ab-4f24-89cc-1332a4233318-kube-api-access-qfzz7" (OuterVolumeSpecName: "kube-api-access-qfzz7") pod "cd832f51-24ab-4f24-89cc-1332a4233318" (UID: "cd832f51-24ab-4f24-89cc-1332a4233318"). InnerVolumeSpecName "kube-api-access-qfzz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:00:58.413433 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.413328 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8-kube-api-access-2j7ff" (OuterVolumeSpecName: "kube-api-access-2j7ff") pod "dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8" (UID: "dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8"). InnerVolumeSpecName "kube-api-access-2j7ff". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:00:58.513937 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.513903 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfzz7\" (UniqueName: \"kubernetes.io/projected/cd832f51-24ab-4f24-89cc-1332a4233318-kube-api-access-qfzz7\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:58.513937 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.513932 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2j7ff\" (UniqueName: \"kubernetes.io/projected/dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8-kube-api-access-2j7ff\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:00:58.756726 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.756693 2560 generic.go:356] "Generic (PLEG): container finished" podID="dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8" containerID="46efae517e36d931c08c037ff80aa920b3975b68701754dde71d0c58f1110acc" exitCode=0 Jul 24 01:00:58.756916 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.756737 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" Jul 24 01:00:58.756916 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.756779 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" event={"ID":"dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8","Type":"ContainerDied","Data":"46efae517e36d931c08c037ff80aa920b3975b68701754dde71d0c58f1110acc"} Jul 24 01:00:58.756916 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.756838 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" event={"ID":"dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8","Type":"ContainerDied","Data":"b902aed322d22da50bf3cff8101e7d76fbae6a7ff23efc81232cecc7d8cbc142"} Jul 24 01:00:58.756916 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.756872 2560 scope.go:117] "RemoveContainer" containerID="46efae517e36d931c08c037ff80aa920b3975b68701754dde71d0c58f1110acc" Jul 24 01:00:58.758182 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.758156 2560 generic.go:356] "Generic (PLEG): container finished" podID="cd832f51-24ab-4f24-89cc-1332a4233318" containerID="7b003074d2d34b6fbe146a6fe599002eaacefae79dbe8d3d2054c615c0105273" exitCode=0 Jul 24 01:00:58.758272 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.758197 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" event={"ID":"cd832f51-24ab-4f24-89cc-1332a4233318","Type":"ContainerDied","Data":"7b003074d2d34b6fbe146a6fe599002eaacefae79dbe8d3d2054c615c0105273"} Jul 24 01:00:58.758272 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.758262 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" event={"ID":"cd832f51-24ab-4f24-89cc-1332a4233318","Type":"ContainerDied","Data":"d81d2fb43131ad87f45d60449897de4935da8f343b78b20dab5c8d6bf48b14e9"} Jul 24 01:00:58.758358 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.758209 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" Jul 24 01:00:58.766363 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.766338 2560 scope.go:117] "RemoveContainer" containerID="46efae517e36d931c08c037ff80aa920b3975b68701754dde71d0c58f1110acc" Jul 24 01:00:58.766749 ip-10-0-139-82 kubenswrapper[2560]: E0724 01:00:58.766727 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46efae517e36d931c08c037ff80aa920b3975b68701754dde71d0c58f1110acc\": container with ID starting with 46efae517e36d931c08c037ff80aa920b3975b68701754dde71d0c58f1110acc not found: ID does not exist" containerID="46efae517e36d931c08c037ff80aa920b3975b68701754dde71d0c58f1110acc" Jul 24 01:00:58.766871 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.766754 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46efae517e36d931c08c037ff80aa920b3975b68701754dde71d0c58f1110acc"} err="failed to get container status \"46efae517e36d931c08c037ff80aa920b3975b68701754dde71d0c58f1110acc\": rpc error: code = NotFound desc = could not find container \"46efae517e36d931c08c037ff80aa920b3975b68701754dde71d0c58f1110acc\": container with ID starting with 46efae517e36d931c08c037ff80aa920b3975b68701754dde71d0c58f1110acc not found: ID does not exist" Jul 24 01:00:58.766871 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.766772 2560 scope.go:117] "RemoveContainer" containerID="7b003074d2d34b6fbe146a6fe599002eaacefae79dbe8d3d2054c615c0105273" Jul 24 01:00:58.775031 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.775011 2560 scope.go:117] "RemoveContainer" containerID="7b003074d2d34b6fbe146a6fe599002eaacefae79dbe8d3d2054c615c0105273" Jul 24 01:00:58.775285 ip-10-0-139-82 kubenswrapper[2560]: E0724 01:00:58.775269 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b003074d2d34b6fbe146a6fe599002eaacefae79dbe8d3d2054c615c0105273\": container with ID starting with 7b003074d2d34b6fbe146a6fe599002eaacefae79dbe8d3d2054c615c0105273 not found: ID does not exist" containerID="7b003074d2d34b6fbe146a6fe599002eaacefae79dbe8d3d2054c615c0105273" Jul 24 01:00:58.775352 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.775294 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b003074d2d34b6fbe146a6fe599002eaacefae79dbe8d3d2054c615c0105273"} err="failed to get container status \"7b003074d2d34b6fbe146a6fe599002eaacefae79dbe8d3d2054c615c0105273\": rpc error: code = NotFound desc = could not find container \"7b003074d2d34b6fbe146a6fe599002eaacefae79dbe8d3d2054c615c0105273\": container with ID starting with 7b003074d2d34b6fbe146a6fe599002eaacefae79dbe8d3d2054c615c0105273 not found: ID does not exist" Jul 24 01:00:58.781754 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.781729 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn"] Jul 24 01:00:58.787833 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.787810 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn"] Jul 24 01:00:58.797412 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.797384 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v"] Jul 24 01:00:58.802651 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:58.802616 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v"] Jul 24 01:00:59.323199 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:59.323154 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-44952a-866fd85ff6-7bz6v" podUID="dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.55:9004: i/o timeout" Jul 24 01:00:59.323583 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:00:59.323162 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-44952a-6465879fc4-lnqcn" podUID="cd832f51-24ab-4f24-89cc-1332a4233318" containerName="payload-processing" probeResult="failure" output="dial tcp 10.134.0.56:9004: i/o timeout" Jul 24 01:01:00.348706 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:00.348668 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd832f51-24ab-4f24-89cc-1332a4233318" path="/var/lib/kubelet/pods/cd832f51-24ab-4f24-89cc-1332a4233318/volumes" Jul 24 01:01:00.349107 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:00.349047 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8" path="/var/lib/kubelet/pods/dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8/volumes" Jul 24 01:01:02.581973 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:02.581938 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:02.952936 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:02.952848 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:08.705351 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:08.705316 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:08.806186 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:08.806143 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:08.974095 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:08.974015 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:09.232995 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.232910 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:09.342031 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.341994 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz"] Jul 24 01:01:09.342569 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342549 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerName="istio-proxy" Jul 24 01:01:09.342620 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342571 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerName="istio-proxy" Jul 24 01:01:09.342620 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342583 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="323ab7ae-1d94-42cc-bf67-914ba3033fde" containerName="payload-pre-processing" Jul 24 01:01:09.342620 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342589 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="323ab7ae-1d94-42cc-bf67-914ba3033fde" containerName="payload-pre-processing" Jul 24 01:01:09.342620 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342604 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd832f51-24ab-4f24-89cc-1332a4233318" containerName="payload-processing" Jul 24 01:01:09.342620 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342610 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd832f51-24ab-4f24-89cc-1332a4233318" containerName="payload-processing" Jul 24 01:01:09.342620 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342618 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83dd244e-6328-49e0-862d-29492c8d1121" containerName="payload-processing" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342624 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="83dd244e-6328-49e0-862d-29492c8d1121" containerName="payload-processing" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342630 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerName="istio-proxy" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342635 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerName="istio-proxy" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342660 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12480620-2261-4624-b53b-d454b97cce3c" containerName="istio-proxy" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342666 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="12480620-2261-4624-b53b-d454b97cce3c" containerName="istio-proxy" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342674 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8" containerName="payload-pre-processing" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342680 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8" containerName="payload-pre-processing" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342685 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12480620-2261-4624-b53b-d454b97cce3c" containerName="istio-proxy" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342690 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="12480620-2261-4624-b53b-d454b97cce3c" containerName="istio-proxy" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342699 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2053d5b-c798-44d9-baf8-461bf1ea3b7a" containerName="payload-pre-processing" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342704 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2053d5b-c798-44d9-baf8-461bf1ea3b7a" containerName="payload-pre-processing" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342763 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2053d5b-c798-44d9-baf8-461bf1ea3b7a" containerName="payload-pre-processing" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342771 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd832f51-24ab-4f24-89cc-1332a4233318" containerName="payload-processing" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342779 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="323ab7ae-1d94-42cc-bf67-914ba3033fde" containerName="payload-pre-processing" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342783 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="12480620-2261-4624-b53b-d454b97cce3c" containerName="istio-proxy" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342791 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbb4a18c-f4b7-4f89-aa79-a32fe254e9a8" containerName="payload-pre-processing" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342798 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerName="istio-proxy" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342803 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="83dd244e-6328-49e0-862d-29492c8d1121" containerName="payload-processing" Jul 24 01:01:09.342854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.342809 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="edf2518a-9c33-4160-ad0f-ffbafc54c2d9" containerName="istio-proxy" Jul 24 01:01:09.347468 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.347447 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz" Jul 24 01:01:09.350170 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.350137 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 01:01:09.350347 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.350198 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-865ss\"" Jul 24 01:01:09.351787 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.351760 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz"] Jul 24 01:01:09.421103 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.421060 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bfa3b06-63d6-4286-a6be-24b1f77d5200-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz\" (UID: \"6bfa3b06-63d6-4286-a6be-24b1f77d5200\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz" Jul 24 01:01:09.522838 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.522803 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bfa3b06-63d6-4286-a6be-24b1f77d5200-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz\" (UID: \"6bfa3b06-63d6-4286-a6be-24b1f77d5200\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz" Jul 24 01:01:09.523434 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.523415 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bfa3b06-63d6-4286-a6be-24b1f77d5200-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz\" (UID: \"6bfa3b06-63d6-4286-a6be-24b1f77d5200\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz" Jul 24 01:01:09.660371 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.660331 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz" Jul 24 01:01:09.806528 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.806500 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz"] Jul 24 01:01:09.809001 ip-10-0-139-82 kubenswrapper[2560]: W0724 01:01:09.808972 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bfa3b06_63d6_4286_a6be_24b1f77d5200.slice/crio-79b323fc8b0bfb21de1591b2ddd7a3af920cc4274db77a1f76cb1b28f01e1f05 WatchSource:0}: Error finding container 79b323fc8b0bfb21de1591b2ddd7a3af920cc4274db77a1f76cb1b28f01e1f05: Status 404 returned error can't find the container with id 79b323fc8b0bfb21de1591b2ddd7a3af920cc4274db77a1f76cb1b28f01e1f05 Jul 24 01:01:09.816209 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:09.816178 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz" event={"ID":"6bfa3b06-63d6-4286-a6be-24b1f77d5200","Type":"ContainerStarted","Data":"79b323fc8b0bfb21de1591b2ddd7a3af920cc4274db77a1f76cb1b28f01e1f05"} Jul 24 01:01:10.822626 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:10.822592 2560 generic.go:356] "Generic (PLEG): container finished" podID="6bfa3b06-63d6-4286-a6be-24b1f77d5200" containerID="b0ed1d9dd1ae059a81d276d64ba8f70add2e19d440be348c8a719fe6403945ef" exitCode=0 Jul 24 01:01:10.823041 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:10.822674 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz" event={"ID":"6bfa3b06-63d6-4286-a6be-24b1f77d5200","Type":"ContainerDied","Data":"b0ed1d9dd1ae059a81d276d64ba8f70add2e19d440be348c8a719fe6403945ef"} Jul 24 01:01:11.970654 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:11.970619 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz" Jul 24 01:01:12.047005 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:12.046971 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bfa3b06-63d6-4286-a6be-24b1f77d5200-maas-api-service-ca\") pod \"6bfa3b06-63d6-4286-a6be-24b1f77d5200\" (UID: \"6bfa3b06-63d6-4286-a6be-24b1f77d5200\") " Jul 24 01:01:12.047384 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:12.047358 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bfa3b06-63d6-4286-a6be-24b1f77d5200-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6bfa3b06-63d6-4286-a6be-24b1f77d5200" (UID: "6bfa3b06-63d6-4286-a6be-24b1f77d5200"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 01:01:12.148261 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:12.148169 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bfa3b06-63d6-4286-a6be-24b1f77d5200-maas-api-service-ca\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:01:12.834708 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:12.834606 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz" Jul 24 01:01:12.834708 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:12.834684 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz" event={"ID":"6bfa3b06-63d6-4286-a6be-24b1f77d5200","Type":"ContainerDied","Data":"79b323fc8b0bfb21de1591b2ddd7a3af920cc4274db77a1f76cb1b28f01e1f05"} Jul 24 01:01:12.834895 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:12.834732 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79b323fc8b0bfb21de1591b2ddd7a3af920cc4274db77a1f76cb1b28f01e1f05" Jul 24 01:01:19.760620 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:19.760579 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:19.795597 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:19.795558 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn"] Jul 24 01:01:19.795868 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:19.795841 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" podUID="93235e8c-eaf6-4484-a413-c0bd7f800f1f" containerName="payload-processing" containerID="cri-o://2516aa50eb42f39e683c7c48d013770068d4a1eef125492733c8eea73b3ecb20" gracePeriod=30 Jul 24 01:01:20.071096 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.071064 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" Jul 24 01:01:20.227825 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.227791 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93235e8c-eaf6-4484-a413-c0bd7f800f1f-plugins-config-volume\") pod \"93235e8c-eaf6-4484-a413-c0bd7f800f1f\" (UID: \"93235e8c-eaf6-4484-a413-c0bd7f800f1f\") " Jul 24 01:01:20.228033 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.227896 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfghm\" (UniqueName: \"kubernetes.io/projected/93235e8c-eaf6-4484-a413-c0bd7f800f1f-kube-api-access-bfghm\") pod \"93235e8c-eaf6-4484-a413-c0bd7f800f1f\" (UID: \"93235e8c-eaf6-4484-a413-c0bd7f800f1f\") " Jul 24 01:01:20.228260 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.228234 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93235e8c-eaf6-4484-a413-c0bd7f800f1f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "93235e8c-eaf6-4484-a413-c0bd7f800f1f" (UID: "93235e8c-eaf6-4484-a413-c0bd7f800f1f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 01:01:20.230079 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.230049 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93235e8c-eaf6-4484-a413-c0bd7f800f1f-kube-api-access-bfghm" (OuterVolumeSpecName: "kube-api-access-bfghm") pod "93235e8c-eaf6-4484-a413-c0bd7f800f1f" (UID: "93235e8c-eaf6-4484-a413-c0bd7f800f1f"). InnerVolumeSpecName "kube-api-access-bfghm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:01:20.329079 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.328982 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93235e8c-eaf6-4484-a413-c0bd7f800f1f-plugins-config-volume\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:01:20.329079 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.329019 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfghm\" (UniqueName: \"kubernetes.io/projected/93235e8c-eaf6-4484-a413-c0bd7f800f1f-kube-api-access-bfghm\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:01:20.884292 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.884259 2560 generic.go:356] "Generic (PLEG): container finished" podID="93235e8c-eaf6-4484-a413-c0bd7f800f1f" containerID="2516aa50eb42f39e683c7c48d013770068d4a1eef125492733c8eea73b3ecb20" exitCode=0 Jul 24 01:01:20.884775 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.884304 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" Jul 24 01:01:20.884775 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.884318 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" event={"ID":"93235e8c-eaf6-4484-a413-c0bd7f800f1f","Type":"ContainerDied","Data":"2516aa50eb42f39e683c7c48d013770068d4a1eef125492733c8eea73b3ecb20"} Jul 24 01:01:20.884775 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.884362 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn" event={"ID":"93235e8c-eaf6-4484-a413-c0bd7f800f1f","Type":"ContainerDied","Data":"44b8cecae6151f42cd558faf44077a37ea9570d35384d6a4735fd092c9794e9c"} Jul 24 01:01:20.884775 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.884384 2560 scope.go:117] "RemoveContainer" containerID="2516aa50eb42f39e683c7c48d013770068d4a1eef125492733c8eea73b3ecb20" Jul 24 01:01:20.893719 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.893697 2560 scope.go:117] "RemoveContainer" containerID="2516aa50eb42f39e683c7c48d013770068d4a1eef125492733c8eea73b3ecb20" Jul 24 01:01:20.894042 ip-10-0-139-82 kubenswrapper[2560]: E0724 01:01:20.894021 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2516aa50eb42f39e683c7c48d013770068d4a1eef125492733c8eea73b3ecb20\": container with ID starting with 2516aa50eb42f39e683c7c48d013770068d4a1eef125492733c8eea73b3ecb20 not found: ID does not exist" containerID="2516aa50eb42f39e683c7c48d013770068d4a1eef125492733c8eea73b3ecb20" Jul 24 01:01:20.894088 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.894051 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2516aa50eb42f39e683c7c48d013770068d4a1eef125492733c8eea73b3ecb20"} err="failed to get container status \"2516aa50eb42f39e683c7c48d013770068d4a1eef125492733c8eea73b3ecb20\": rpc error: code = NotFound desc = could not find container \"2516aa50eb42f39e683c7c48d013770068d4a1eef125492733c8eea73b3ecb20\": container with ID starting with 2516aa50eb42f39e683c7c48d013770068d4a1eef125492733c8eea73b3ecb20 not found: ID does not exist" Jul 24 01:01:20.900436 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.900402 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn"] Jul 24 01:01:20.904941 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:20.904914 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-91a75b-6dd77dc47d-bprtn"] Jul 24 01:01:22.339040 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:22.339005 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:22.347935 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:22.347908 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93235e8c-eaf6-4484-a413-c0bd7f800f1f" path="/var/lib/kubelet/pods/93235e8c-eaf6-4484-a413-c0bd7f800f1f/volumes" Jul 24 01:01:22.392934 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:22.392900 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:24.415590 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:24.415555 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz"] Jul 24 01:01:24.417704 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:24.417680 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-91a75b-8jngz"] Jul 24 01:01:24.419201 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:24.419174 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:24.795684 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:24.795632 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:26.348164 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:26.348130 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bfa3b06-63d6-4286-a6be-24b1f77d5200" path="/var/lib/kubelet/pods/6bfa3b06-63d6-4286-a6be-24b1f77d5200/volumes" Jul 24 01:01:30.641869 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:30.641838 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:30.812902 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:30.812868 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:30.909668 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:30.909563 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v"] Jul 24 01:01:30.909822 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:30.909797 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" podUID="a3007cad-2bda-4101-af9d-9bf95d19c099" containerName="istio-proxy" containerID="cri-o://a31834ccafad6d0b51ac5046be31f6ed422688288a21fabda5933017b7127c14" gracePeriod=30 Jul 24 01:01:30.926278 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:30.926248 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:31.154033 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:31.153998 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:36.161989 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.161962 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 01:01:36.183112 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183016 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-credential-socket\") pod \"a3007cad-2bda-4101-af9d-9bf95d19c099\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " Jul 24 01:01:36.183112 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183072 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-workload-socket\") pod \"a3007cad-2bda-4101-af9d-9bf95d19c099\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " Jul 24 01:01:36.183112 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183097 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-data\") pod \"a3007cad-2bda-4101-af9d-9bf95d19c099\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " Jul 24 01:01:36.183422 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183146 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-token\") pod \"a3007cad-2bda-4101-af9d-9bf95d19c099\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " Jul 24 01:01:36.183422 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183192 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-envoy\") pod \"a3007cad-2bda-4101-af9d-9bf95d19c099\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " Jul 24 01:01:36.183422 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183208 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3007cad-2bda-4101-af9d-9bf95d19c099-istiod-ca-cert\") pod \"a3007cad-2bda-4101-af9d-9bf95d19c099\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " Jul 24 01:01:36.183422 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183238 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hptkr\" (UniqueName: \"kubernetes.io/projected/a3007cad-2bda-4101-af9d-9bf95d19c099-kube-api-access-hptkr\") pod \"a3007cad-2bda-4101-af9d-9bf95d19c099\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " Jul 24 01:01:36.183422 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183263 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-workload-certs\") pod \"a3007cad-2bda-4101-af9d-9bf95d19c099\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " Jul 24 01:01:36.183422 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183331 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a3007cad-2bda-4101-af9d-9bf95d19c099" (UID: "a3007cad-2bda-4101-af9d-9bf95d19c099"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 01:01:36.185258 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183411 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a3007cad-2bda-4101-af9d-9bf95d19c099" (UID: "a3007cad-2bda-4101-af9d-9bf95d19c099"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 01:01:36.185258 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183446 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-podinfo\") pod \"a3007cad-2bda-4101-af9d-9bf95d19c099\" (UID: \"a3007cad-2bda-4101-af9d-9bf95d19c099\") " Jul 24 01:01:36.185258 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183557 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a3007cad-2bda-4101-af9d-9bf95d19c099" (UID: "a3007cad-2bda-4101-af9d-9bf95d19c099"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 01:01:36.185258 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183621 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3007cad-2bda-4101-af9d-9bf95d19c099-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a3007cad-2bda-4101-af9d-9bf95d19c099" (UID: "a3007cad-2bda-4101-af9d-9bf95d19c099"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 01:01:36.185258 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183946 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-workload-socket\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:01:36.185258 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183969 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3007cad-2bda-4101-af9d-9bf95d19c099-istiod-ca-cert\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:01:36.185258 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183983 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-workload-certs\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:01:36.185258 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.183996 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-credential-socket\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:01:36.186165 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.185500 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-data" (OuterVolumeSpecName: "istio-data") pod "a3007cad-2bda-4101-af9d-9bf95d19c099" (UID: "a3007cad-2bda-4101-af9d-9bf95d19c099"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 01:01:36.186681 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.186607 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a3007cad-2bda-4101-af9d-9bf95d19c099" (UID: "a3007cad-2bda-4101-af9d-9bf95d19c099"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 01:01:36.187141 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.187105 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-token" (OuterVolumeSpecName: "istio-token") pod "a3007cad-2bda-4101-af9d-9bf95d19c099" (UID: "a3007cad-2bda-4101-af9d-9bf95d19c099"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:01:36.187598 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.187369 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a3007cad-2bda-4101-af9d-9bf95d19c099" (UID: "a3007cad-2bda-4101-af9d-9bf95d19c099"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 01:01:36.187914 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.187887 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3007cad-2bda-4101-af9d-9bf95d19c099-kube-api-access-hptkr" (OuterVolumeSpecName: "kube-api-access-hptkr") pod "a3007cad-2bda-4101-af9d-9bf95d19c099" (UID: "a3007cad-2bda-4101-af9d-9bf95d19c099"). InnerVolumeSpecName "kube-api-access-hptkr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:01:36.285050 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.285019 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-podinfo\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:01:36.285050 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.285047 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-data\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:01:36.285050 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.285055 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-token\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:01:36.285289 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.285063 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3007cad-2bda-4101-af9d-9bf95d19c099-istio-envoy\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:01:36.285289 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.285071 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hptkr\" (UniqueName: \"kubernetes.io/projected/a3007cad-2bda-4101-af9d-9bf95d19c099-kube-api-access-hptkr\") on node \"ip-10-0-139-82.ec2.internal\" DevicePath \"\"" Jul 24 01:01:36.966329 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.966291 2560 generic.go:356] "Generic (PLEG): container finished" podID="a3007cad-2bda-4101-af9d-9bf95d19c099" containerID="a31834ccafad6d0b51ac5046be31f6ed422688288a21fabda5933017b7127c14" exitCode=0 Jul 24 01:01:36.966545 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.966344 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" event={"ID":"a3007cad-2bda-4101-af9d-9bf95d19c099","Type":"ContainerDied","Data":"a31834ccafad6d0b51ac5046be31f6ed422688288a21fabda5933017b7127c14"} Jul 24 01:01:36.966545 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.966367 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" Jul 24 01:01:36.966545 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.966390 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v" event={"ID":"a3007cad-2bda-4101-af9d-9bf95d19c099","Type":"ContainerDied","Data":"46f5363d610ead8845d9860dc949bbb219acae609ce7a584cee470689b0c99ef"} Jul 24 01:01:36.966545 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.966413 2560 scope.go:117] "RemoveContainer" containerID="a31834ccafad6d0b51ac5046be31f6ed422688288a21fabda5933017b7127c14" Jul 24 01:01:36.975609 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.975592 2560 scope.go:117] "RemoveContainer" containerID="e1c1663be9217217bba842391b3705daad24eebf785000d8fa7e1b07ead1535b" Jul 24 01:01:36.984580 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.984555 2560 scope.go:117] "RemoveContainer" containerID="a31834ccafad6d0b51ac5046be31f6ed422688288a21fabda5933017b7127c14" Jul 24 01:01:36.984950 ip-10-0-139-82 kubenswrapper[2560]: E0724 01:01:36.984925 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a31834ccafad6d0b51ac5046be31f6ed422688288a21fabda5933017b7127c14\": container with ID starting with a31834ccafad6d0b51ac5046be31f6ed422688288a21fabda5933017b7127c14 not found: ID does not exist" containerID="a31834ccafad6d0b51ac5046be31f6ed422688288a21fabda5933017b7127c14" Jul 24 01:01:36.985017 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.984963 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a31834ccafad6d0b51ac5046be31f6ed422688288a21fabda5933017b7127c14"} err="failed to get container status \"a31834ccafad6d0b51ac5046be31f6ed422688288a21fabda5933017b7127c14\": rpc error: code = NotFound desc = could not find container \"a31834ccafad6d0b51ac5046be31f6ed422688288a21fabda5933017b7127c14\": container with ID starting with a31834ccafad6d0b51ac5046be31f6ed422688288a21fabda5933017b7127c14 not found: ID does not exist" Jul 24 01:01:36.985017 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.984988 2560 scope.go:117] "RemoveContainer" containerID="e1c1663be9217217bba842391b3705daad24eebf785000d8fa7e1b07ead1535b" Jul 24 01:01:36.985276 ip-10-0-139-82 kubenswrapper[2560]: E0724 01:01:36.985258 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1c1663be9217217bba842391b3705daad24eebf785000d8fa7e1b07ead1535b\": container with ID starting with e1c1663be9217217bba842391b3705daad24eebf785000d8fa7e1b07ead1535b not found: ID does not exist" containerID="e1c1663be9217217bba842391b3705daad24eebf785000d8fa7e1b07ead1535b" Jul 24 01:01:36.985322 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.985284 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1c1663be9217217bba842391b3705daad24eebf785000d8fa7e1b07ead1535b"} err="failed to get container status \"e1c1663be9217217bba842391b3705daad24eebf785000d8fa7e1b07ead1535b\": rpc error: code = NotFound desc = could not find container \"e1c1663be9217217bba842391b3705daad24eebf785000d8fa7e1b07ead1535b\": container with ID starting with e1c1663be9217217bba842391b3705daad24eebf785000d8fa7e1b07ead1535b not found: ID does not exist" Jul 24 01:01:36.986788 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.986769 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v"] Jul 24 01:01:36.991507 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:36.991476 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-91a75b-openshift-default-fbc8dcdd4-xdr9v"] Jul 24 01:01:37.858738 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:37.858711 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cdb55f47c-jwbd6_8cb4e2fd-ff73-4aed-a48b-6ec842c744ea/manager/0.log" Jul 24 01:01:37.865039 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:37.865015 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cdb55f47c-jwbd6_8cb4e2fd-ff73-4aed-a48b-6ec842c744ea/copy-manifests/0.log" Jul 24 01:01:38.347852 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:38.347818 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3007cad-2bda-4101-af9d-9bf95d19c099" path="/var/lib/kubelet/pods/a3007cad-2bda-4101-af9d-9bf95d19c099/volumes" Jul 24 01:01:38.474457 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:38.474429 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747550-69wkt_9e4514ed-74e4-4b15-a6c0-d07c42d15a31/cleanup/0.log" Jul 24 01:01:38.587627 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:38.587600 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747565-chpbg_bbb7bea3-e6d7-418e-80b8-8867be708972/cleanup/0.log" Jul 24 01:01:41.052382 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:41.052342 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-n9vgp_a0b4d128-f563-4479-a832-b5eee7594d8d/manager/0.log" Jul 24 01:01:41.411854 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:41.411777 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc_4d50a234-1c26-4737-afe6-c88f47b7e388/istio-proxy/0.log" Jul 24 01:01:41.533057 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:41.533023 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wv7kg_fd4f15a3-8fd1-4602-80f6-612ae7d031dd/discovery/0.log" Jul 24 01:01:42.715883 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:42.715855 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vrvxw_e78a4eac-43c4-455a-a37c-ba3dee8e24eb/main/0.log" Jul 24 01:01:42.828329 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:42.828282 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-rrpn7_8d800604-60f1-4cda-9aaf-92a4d6a8864c/main/0.log" Jul 24 01:01:48.528060 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:48.528020 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:48.627402 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:48.627370 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" limit=1024 Jul 24 01:01:54.245579 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:54.245547 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k66hc_ff0d6407-0175-4c0e-a54c-528dae80387a/global-pull-secret-syncer/0.log" Jul 24 01:01:54.297890 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:54.297857 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kfmld_87ccd406-4359-435e-8c88-9e677b0da0ec/konnectivity-agent/0.log" Jul 24 01:01:54.418178 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:54.418145 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-82.ec2.internal_dd7dabc7b022df6b0fbb3ed250311918/haproxy/0.log" Jul 24 01:01:59.930103 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:01:59.930069 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-n9vgp_a0b4d128-f563-4479-a832-b5eee7594d8d/manager/0.log" Jul 24 01:02:01.595517 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:01.595488 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7d6efdc7-3622-4000-84a4-bfd29e1bdc45/alertmanager/0.log" Jul 24 01:02:01.618138 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:01.618109 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7d6efdc7-3622-4000-84a4-bfd29e1bdc45/config-reloader/0.log" Jul 24 01:02:01.642467 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:01.642394 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7d6efdc7-3622-4000-84a4-bfd29e1bdc45/kube-rbac-proxy-web/0.log" Jul 24 01:02:01.663736 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:01.663706 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7d6efdc7-3622-4000-84a4-bfd29e1bdc45/kube-rbac-proxy/0.log" Jul 24 01:02:01.687468 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:01.687440 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7d6efdc7-3622-4000-84a4-bfd29e1bdc45/kube-rbac-proxy-metric/0.log" Jul 24 01:02:01.709048 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:01.709018 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7d6efdc7-3622-4000-84a4-bfd29e1bdc45/prom-label-proxy/0.log" Jul 24 01:02:01.731752 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:01.731729 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7d6efdc7-3622-4000-84a4-bfd29e1bdc45/init-config-reloader/0.log" Jul 24 01:02:01.801487 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:01.801463 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-nhxgt_4acaa63f-fc09-4074-9806-f9e9a5b43142/kube-state-metrics/0.log" Jul 24 01:02:01.822981 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:01.822953 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-nhxgt_4acaa63f-fc09-4074-9806-f9e9a5b43142/kube-rbac-proxy-main/0.log" Jul 24 01:02:01.845966 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:01.845940 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-nhxgt_4acaa63f-fc09-4074-9806-f9e9a5b43142/kube-rbac-proxy-self/0.log" Jul 24 01:02:01.873754 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:01.873727 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6d8f567488-7cnck_b164d179-d56a-451e-9b4e-00b001172f51/metrics-server/0.log" Jul 24 01:02:01.900936 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:01.900862 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-rvvkk_1cf6076a-b8f1-44aa-a565-28328b56a8d9/monitoring-plugin/0.log" Jul 24 01:02:02.119249 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.119217 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vsn7g_ecc5f68e-2d88-4dde-809b-c0654937b597/node-exporter/0.log" Jul 24 01:02:02.139528 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.139502 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vsn7g_ecc5f68e-2d88-4dde-809b-c0654937b597/kube-rbac-proxy/0.log" Jul 24 01:02:02.160660 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.160566 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vsn7g_ecc5f68e-2d88-4dde-809b-c0654937b597/init-textfile/0.log" Jul 24 01:02:02.187053 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.187027 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-rdchn_da7c354c-8bc2-4e55-a070-d49f71cd4303/kube-rbac-proxy-main/0.log" Jul 24 01:02:02.207365 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.207343 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-rdchn_da7c354c-8bc2-4e55-a070-d49f71cd4303/kube-rbac-proxy-self/0.log" Jul 24 01:02:02.230559 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.230534 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-rdchn_da7c354c-8bc2-4e55-a070-d49f71cd4303/openshift-state-metrics/0.log" Jul 24 01:02:02.277176 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.277147 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_41ebf30d-ef1a-440f-b188-cb96f910a5a4/prometheus/0.log" Jul 24 01:02:02.300480 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.300453 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_41ebf30d-ef1a-440f-b188-cb96f910a5a4/config-reloader/0.log" Jul 24 01:02:02.321897 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.321874 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_41ebf30d-ef1a-440f-b188-cb96f910a5a4/thanos-sidecar/0.log" Jul 24 01:02:02.344562 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.344536 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_41ebf30d-ef1a-440f-b188-cb96f910a5a4/kube-rbac-proxy-web/0.log" Jul 24 01:02:02.366917 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.366886 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_41ebf30d-ef1a-440f-b188-cb96f910a5a4/kube-rbac-proxy/0.log" Jul 24 01:02:02.388784 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.388753 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_41ebf30d-ef1a-440f-b188-cb96f910a5a4/kube-rbac-proxy-thanos/0.log" Jul 24 01:02:02.409764 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.409734 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_41ebf30d-ef1a-440f-b188-cb96f910a5a4/init-config-reloader/0.log" Jul 24 01:02:02.440342 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.440264 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-hmvqn_d8355e71-b509-4932-9797-afcaded7f8fe/prometheus-operator/0.log" Jul 24 01:02:02.459433 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.459403 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-hmvqn_d8355e71-b509-4932-9797-afcaded7f8fe/kube-rbac-proxy/0.log" Jul 24 01:02:02.510904 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.510873 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-577b855d68-tp4ph_f54ac913-2a49-4d70-adfc-2fa67aadf7ed/telemeter-client/0.log" Jul 24 01:02:02.532246 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.532222 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-577b855d68-tp4ph_f54ac913-2a49-4d70-adfc-2fa67aadf7ed/reload/0.log" Jul 24 01:02:02.553797 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.553769 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-577b855d68-tp4ph_f54ac913-2a49-4d70-adfc-2fa67aadf7ed/kube-rbac-proxy/0.log" Jul 24 01:02:02.583681 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.583635 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bcd874fcc-r2sb2_6819273d-e59d-4321-b9f3-e3afd59cf648/thanos-query/0.log" Jul 24 01:02:02.606958 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.606930 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bcd874fcc-r2sb2_6819273d-e59d-4321-b9f3-e3afd59cf648/kube-rbac-proxy-web/0.log" Jul 24 01:02:02.634143 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.634117 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bcd874fcc-r2sb2_6819273d-e59d-4321-b9f3-e3afd59cf648/kube-rbac-proxy/0.log" Jul 24 01:02:02.654938 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.654908 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bcd874fcc-r2sb2_6819273d-e59d-4321-b9f3-e3afd59cf648/prom-label-proxy/0.log" Jul 24 01:02:02.675898 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.675870 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bcd874fcc-r2sb2_6819273d-e59d-4321-b9f3-e3afd59cf648/kube-rbac-proxy-rules/0.log" Jul 24 01:02:02.697173 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:02.697099 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bcd874fcc-r2sb2_6819273d-e59d-4321-b9f3-e3afd59cf648/kube-rbac-proxy-metrics/0.log" Jul 24 01:02:04.457922 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.457843 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk"] Jul 24 01:02:04.458297 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.458230 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3007cad-2bda-4101-af9d-9bf95d19c099" containerName="istio-proxy" Jul 24 01:02:04.458297 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.458241 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3007cad-2bda-4101-af9d-9bf95d19c099" containerName="istio-proxy" Jul 24 01:02:04.458297 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.458261 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3007cad-2bda-4101-af9d-9bf95d19c099" containerName="istio-proxy" Jul 24 01:02:04.458297 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.458267 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3007cad-2bda-4101-af9d-9bf95d19c099" containerName="istio-proxy" Jul 24 01:02:04.458297 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.458279 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bfa3b06-63d6-4286-a6be-24b1f77d5200" containerName="revoke-keys" Jul 24 01:02:04.458297 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.458286 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bfa3b06-63d6-4286-a6be-24b1f77d5200" containerName="revoke-keys" Jul 24 01:02:04.458297 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.458297 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93235e8c-eaf6-4484-a413-c0bd7f800f1f" containerName="payload-processing" Jul 24 01:02:04.458519 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.458303 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="93235e8c-eaf6-4484-a413-c0bd7f800f1f" containerName="payload-processing" Jul 24 01:02:04.458519 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.458358 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="12480620-2261-4624-b53b-d454b97cce3c" containerName="istio-proxy" Jul 24 01:02:04.458519 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.458367 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3007cad-2bda-4101-af9d-9bf95d19c099" containerName="istio-proxy" Jul 24 01:02:04.458519 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.458375 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="93235e8c-eaf6-4484-a413-c0bd7f800f1f" containerName="payload-processing" Jul 24 01:02:04.458519 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.458381 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3007cad-2bda-4101-af9d-9bf95d19c099" containerName="istio-proxy" Jul 24 01:02:04.458519 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.458388 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bfa3b06-63d6-4286-a6be-24b1f77d5200" containerName="revoke-keys" Jul 24 01:02:04.461574 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.461557 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.465152 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.465131 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7v8jb\"/\"kube-root-ca.crt\"" Jul 24 01:02:04.466258 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.466233 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7v8jb\"/\"openshift-service-ca.crt\"" Jul 24 01:02:04.466406 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.466297 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7v8jb\"/\"default-dockercfg-hlg9z\"" Jul 24 01:02:04.469684 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.469473 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk"] Jul 24 01:02:04.547526 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.547486 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ptdr\" (UniqueName: \"kubernetes.io/projected/8e0cc996-e397-445a-968f-8b85bae6f1cc-kube-api-access-8ptdr\") pod \"perf-node-gather-daemonset-t6ndk\" (UID: \"8e0cc996-e397-445a-968f-8b85bae6f1cc\") " pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.547526 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.547526 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e0cc996-e397-445a-968f-8b85bae6f1cc-podres\") pod \"perf-node-gather-daemonset-t6ndk\" (UID: \"8e0cc996-e397-445a-968f-8b85bae6f1cc\") " pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.547776 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.547603 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e0cc996-e397-445a-968f-8b85bae6f1cc-sys\") pod \"perf-node-gather-daemonset-t6ndk\" (UID: \"8e0cc996-e397-445a-968f-8b85bae6f1cc\") " pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.547776 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.547733 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e0cc996-e397-445a-968f-8b85bae6f1cc-lib-modules\") pod \"perf-node-gather-daemonset-t6ndk\" (UID: \"8e0cc996-e397-445a-968f-8b85bae6f1cc\") " pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.547776 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.547762 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e0cc996-e397-445a-968f-8b85bae6f1cc-proc\") pod \"perf-node-gather-daemonset-t6ndk\" (UID: \"8e0cc996-e397-445a-968f-8b85bae6f1cc\") " pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.649027 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.648972 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ptdr\" (UniqueName: \"kubernetes.io/projected/8e0cc996-e397-445a-968f-8b85bae6f1cc-kube-api-access-8ptdr\") pod \"perf-node-gather-daemonset-t6ndk\" (UID: \"8e0cc996-e397-445a-968f-8b85bae6f1cc\") " pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.649027 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.649026 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e0cc996-e397-445a-968f-8b85bae6f1cc-podres\") pod \"perf-node-gather-daemonset-t6ndk\" (UID: \"8e0cc996-e397-445a-968f-8b85bae6f1cc\") " pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.649306 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.649055 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e0cc996-e397-445a-968f-8b85bae6f1cc-sys\") pod \"perf-node-gather-daemonset-t6ndk\" (UID: \"8e0cc996-e397-445a-968f-8b85bae6f1cc\") " pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.649306 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.649099 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e0cc996-e397-445a-968f-8b85bae6f1cc-lib-modules\") pod \"perf-node-gather-daemonset-t6ndk\" (UID: \"8e0cc996-e397-445a-968f-8b85bae6f1cc\") " pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.649306 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.649118 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e0cc996-e397-445a-968f-8b85bae6f1cc-proc\") pod \"perf-node-gather-daemonset-t6ndk\" (UID: \"8e0cc996-e397-445a-968f-8b85bae6f1cc\") " pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.649306 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.649191 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e0cc996-e397-445a-968f-8b85bae6f1cc-proc\") pod \"perf-node-gather-daemonset-t6ndk\" (UID: \"8e0cc996-e397-445a-968f-8b85bae6f1cc\") " pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.649306 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.649197 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e0cc996-e397-445a-968f-8b85bae6f1cc-sys\") pod \"perf-node-gather-daemonset-t6ndk\" (UID: \"8e0cc996-e397-445a-968f-8b85bae6f1cc\") " pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.649306 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.649233 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e0cc996-e397-445a-968f-8b85bae6f1cc-podres\") pod \"perf-node-gather-daemonset-t6ndk\" (UID: \"8e0cc996-e397-445a-968f-8b85bae6f1cc\") " pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.649306 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.649255 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e0cc996-e397-445a-968f-8b85bae6f1cc-lib-modules\") pod \"perf-node-gather-daemonset-t6ndk\" (UID: \"8e0cc996-e397-445a-968f-8b85bae6f1cc\") " pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.658698 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.658673 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ptdr\" (UniqueName: \"kubernetes.io/projected/8e0cc996-e397-445a-968f-8b85bae6f1cc-kube-api-access-8ptdr\") pod \"perf-node-gather-daemonset-t6ndk\" (UID: \"8e0cc996-e397-445a-968f-8b85bae6f1cc\") " pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.773565 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.773532 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:04.915094 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:04.911885 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk"] Jul 24 01:02:05.114477 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:05.114380 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" event={"ID":"8e0cc996-e397-445a-968f-8b85bae6f1cc","Type":"ContainerStarted","Data":"1a67868411432a761e3a1125a14da12046a5d7281b9fcbe91027e942a3cd5cef"} Jul 24 01:02:05.114477 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:05.114438 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" event={"ID":"8e0cc996-e397-445a-968f-8b85bae6f1cc","Type":"ContainerStarted","Data":"21daa6f076b09f4cdc4f96be50f9ae87155b3091665567e94599581c9aa65bcb"} Jul 24 01:02:05.114694 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:05.114520 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:05.125798 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:05.125770 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-p88lj_5ff612cc-1d5c-4195-b649-be35a283ce97/download-server/0.log" Jul 24 01:02:05.130985 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:05.130936 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" podStartSLOduration=1.130921487 podStartE2EDuration="1.130921487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 01:02:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:02:05.129796616 +0000 UTC m=+3053.735110935" watchObservedRunningTime="2026-07-24 01:02:05.130921487 +0000 UTC m=+3053.736235797" Jul 24 01:02:06.730277 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:06.730246 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fxm4z_989c9067-460b-4838-b1b0-3da75e2a848f/dns/0.log" Jul 24 01:02:06.751356 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:06.751325 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fxm4z_989c9067-460b-4838-b1b0-3da75e2a848f/kube-rbac-proxy/0.log" Jul 24 01:02:06.920101 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:06.920075 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wb5s8_15937a1b-bb3c-46a5-8b30-3d2e4c260be3/dns-node-resolver/0.log" Jul 24 01:02:07.581956 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:07.581927 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c4c45_567139c0-866a-4ff0-9085-579046198698/node-ca/0.log" Jul 24 01:02:08.561651 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:08.561581 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc_4d50a234-1c26-4737-afe6-c88f47b7e388/istio-proxy/0.log" Jul 24 01:02:08.710101 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:08.710069 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wv7kg_fd4f15a3-8fd1-4602-80f6-612ae7d031dd/discovery/0.log" Jul 24 01:02:09.769089 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:09.769061 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rsb8q_0c87883b-9a82-4049-a788-b28dc9f49340/serve-healthcheck-canary/0.log" Jul 24 01:02:10.326796 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:10.326767 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d7q6r_1a539eef-502e-48e6-8020-2e6d9cc6ff5b/kube-rbac-proxy/0.log" Jul 24 01:02:10.350020 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:10.349992 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d7q6r_1a539eef-502e-48e6-8020-2e6d9cc6ff5b/exporter/0.log" Jul 24 01:02:10.370415 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:10.370389 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d7q6r_1a539eef-502e-48e6-8020-2e6d9cc6ff5b/extractor/0.log" Jul 24 01:02:11.129946 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:11.129916 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7v8jb/perf-node-gather-daemonset-t6ndk" Jul 24 01:02:13.109232 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:13.109205 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cdb55f47c-jwbd6_8cb4e2fd-ff73-4aed-a48b-6ec842c744ea/manager/0.log" Jul 24 01:02:13.131455 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:13.131430 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cdb55f47c-jwbd6_8cb4e2fd-ff73-4aed-a48b-6ec842c744ea/copy-manifests/0.log" Jul 24 01:02:14.604669 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:14.604564 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5bbf775ccb-k2qtk_9fde78b7-bbde-4051-832b-808c7a7af8e1/manager/0.log" Jul 24 01:02:14.679300 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:14.679259 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-tblp4_580b0146-fbb4-4da7-9a4b-ae78662b8e5b/openshift-lws-operator/0.log" Jul 24 01:02:14.811444 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:14.811413 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc_4d50a234-1c26-4737-afe6-c88f47b7e388/istio-proxy/0.log" Jul 24 01:02:14.942209 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:14.942107 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wv7kg_fd4f15a3-8fd1-4602-80f6-612ae7d031dd/discovery/0.log" Jul 24 01:02:16.545164 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:16.545085 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-x2prc_4d50a234-1c26-4737-afe6-c88f47b7e388/istio-proxy/0.log" Jul 24 01:02:16.691947 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:16.691915 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wv7kg_fd4f15a3-8fd1-4602-80f6-612ae7d031dd/discovery/0.log" Jul 24 01:02:21.249916 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:21.249879 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvwnr_e6bdd868-f7ad-4866-965a-d60339268678/kube-multus-additional-cni-plugins/0.log" Jul 24 01:02:21.270922 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:21.270889 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvwnr_e6bdd868-f7ad-4866-965a-d60339268678/egress-router-binary-copy/0.log" Jul 24 01:02:21.291322 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:21.291293 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvwnr_e6bdd868-f7ad-4866-965a-d60339268678/cni-plugins/0.log" Jul 24 01:02:21.310912 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:21.310882 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvwnr_e6bdd868-f7ad-4866-965a-d60339268678/bond-cni-plugin/0.log" Jul 24 01:02:21.330519 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:21.330483 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvwnr_e6bdd868-f7ad-4866-965a-d60339268678/routeoverride-cni/0.log" Jul 24 01:02:21.351993 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:21.351959 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvwnr_e6bdd868-f7ad-4866-965a-d60339268678/whereabouts-cni-bincopy/0.log" Jul 24 01:02:21.372237 ip-10-0-139-82 kubenswrapper[2560]: I0724 01:02:21.372208 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvwnr_e6bdd868-f7ad-4866-965a-d60339268678/whereabouts-cni/0.log"