Jul 07 06:29:53.448114 ip-10-0-130-176 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 07 06:29:53.448124 ip-10-0-130-176 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 07 06:29:53.448130 ip-10-0-130-176 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 07 06:29:53.448405 ip-10-0-130-176 systemd[1]: Failed to start Kubernetes Kubelet. Jul 07 06:30:03.542442 ip-10-0-130-176 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 07 06:30:03.542461 ip-10-0-130-176 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 2ce54e8eba5b4a97bb94280b697cb4e0 -- Jul 07 06:32:04.037646 ip-10-0-130-176 systemd[1]: Starting Kubernetes Kubelet... Jul 07 06:32:04.478252 ip-10-0-130-176 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 06:32:04.478252 ip-10-0-130-176 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 06:32:04.478252 ip-10-0-130-176 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 06:32:04.478252 ip-10-0-130-176 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 06:32:04.478252 ip-10-0-130-176 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 06:32:04.481654 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.479449 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 06:32:04.563917 ip-10-0-130-176 systemd[1]: Started Kubernetes Kubelet. Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492190 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492334 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492342 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492345 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492349 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492352 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492355 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492358 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492361 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492374 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492377 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492380 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492383 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492385 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492388 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492391 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492394 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492396 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492399 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 06:32:04.579090 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492402 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492404 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492407 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492409 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492412 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492415 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492418 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492421 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492423 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492426 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492428 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492431 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492433 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492437 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492442 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492448 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492453 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492456 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492460 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 06:32:04.582665 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492463 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492466 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492469 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492471 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492474 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492477 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492479 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492482 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492484 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492487 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492490 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492493 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492495 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492498 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492501 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492504 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492506 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492509 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492512 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492515 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 06:32:04.583534 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492518 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492520 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492523 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492526 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492528 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492531 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492533 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492536 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492539 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492542 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492544 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492547 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492549 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492555 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492560 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492563 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492566 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492569 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492571 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492574 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 06:32:04.584816 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492577 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492580 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492582 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492585 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492588 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492591 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492594 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492598 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492602 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492605 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492607 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492610 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492613 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492615 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492618 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492620 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492623 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492625 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492628 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 06:32:04.588012 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492631 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.492634 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493433 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493442 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493446 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493449 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493452 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493455 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493457 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493460 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493462 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493465 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493468 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493471 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493473 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493476 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493478 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493481 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493483 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493493 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 06:32:04.588830 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493495 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493498 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493501 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493503 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493506 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493509 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493511 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493514 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493516 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493519 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493522 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493525 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493527 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493530 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493533 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493536 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493538 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493541 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493545 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 06:32:04.589714 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493549 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493552 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493555 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493558 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493560 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493563 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493566 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493569 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493572 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493574 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493576 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493579 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493581 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493589 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493594 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493597 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493599 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493601 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493604 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493607 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 06:32:04.591689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493610 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493613 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493615 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493618 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493620 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493624 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493627 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493629 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493632 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493634 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493637 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493640 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493642 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493645 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493647 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493650 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493653 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493656 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493658 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493661 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 06:32:04.592567 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493663 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493666 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493668 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493670 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493673 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493675 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493680 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493682 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493685 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493687 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493692 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493695 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493698 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493700 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493703 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493706 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493708 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493711 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493713 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 06:32:04.595111 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493717 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493720 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.493723 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494147 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494153 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494156 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494159 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494162 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494166 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494170 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494173 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494175 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494178 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494181 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494183 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494186 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494189 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494191 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494194 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494199 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 06:32:04.595981 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494201 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494204 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494206 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494209 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494212 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494214 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494217 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494219 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494222 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494224 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494227 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494229 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494232 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494234 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494236 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494239 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494241 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494244 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494247 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494249 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 06:32:04.750300 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494252 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494254 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494256 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494259 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494261 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494264 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494266 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494268 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494271 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494273 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494276 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494280 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494283 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494286 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494289 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494291 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494294 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494297 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494299 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494302 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 06:32:04.789298 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494304 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494307 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494309 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494312 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494315 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494318 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494322 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494324 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494327 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494330 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494332 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494335 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494337 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494340 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494342 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494345 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494347 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494349 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494352 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 06:32:04.795874 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494356 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494360 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494379 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494382 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494386 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494388 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494392 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494395 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494397 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494401 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494404 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494406 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494409 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494411 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494414 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494416 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494419 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494421 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494424 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494426 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 06:32:04.797193 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494429 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494431 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.494434 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494503 2579 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494510 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494516 2579 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494520 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494534 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494538 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494542 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494547 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494550 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494553 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494556 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494559 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494562 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494567 2579 flags.go:64] FLAG: --cgroup-root="" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494570 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494573 2579 flags.go:64] FLAG: --client-ca-file="" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494577 2579 flags.go:64] FLAG: --cloud-provider="external" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494580 2579 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494587 2579 flags.go:64] FLAG: --cluster-domain="" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494590 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494593 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 06:32:04.798342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494596 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494599 2579 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494603 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494606 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494609 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494612 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494615 2579 flags.go:64] FLAG: --contention-profiling="false" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494618 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494621 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494624 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494627 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494631 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494634 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494637 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494640 2579 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494643 2579 flags.go:64] FLAG: --enable-server="true" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494645 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494649 2579 flags.go:64] FLAG: --event-burst="100" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494652 2579 flags.go:64] FLAG: --event-qps="50" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494655 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494658 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494660 2579 flags.go:64] FLAG: --eviction-hard="" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494664 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494666 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494671 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 06:32:04.800480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494674 2579 flags.go:64] FLAG: --eviction-soft="" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494677 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494680 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494683 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494687 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494690 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494693 2579 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494696 2579 flags.go:64] FLAG: --feature-gates="" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494700 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494702 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494705 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494708 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494711 2579 flags.go:64] FLAG: --healthz-port="10248" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494714 2579 flags.go:64] FLAG: --help="false" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494717 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-130-176.ec2.internal" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494720 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494723 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494726 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494729 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494732 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494735 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494738 2579 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494740 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 06:32:04.801655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494744 2579 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494746 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494749 2579 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494752 2579 flags.go:64] FLAG: --kube-reserved="" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494755 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494758 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494761 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494763 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494767 2579 flags.go:64] FLAG: --lock-file="" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494770 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494773 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494775 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494780 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494785 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494788 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494791 2579 flags.go:64] FLAG: --logging-format="text" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494794 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494797 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494800 2579 flags.go:64] FLAG: --manifest-url="" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494803 2579 flags.go:64] FLAG: --manifest-url-header="" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494807 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494810 2579 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494814 2579 flags.go:64] FLAG: --max-pods="110" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494817 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494820 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 06:32:04.803046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494822 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494826 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494829 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494831 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494834 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494841 2579 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494844 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494847 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494850 2579 flags.go:64] FLAG: --pod-cidr="" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494853 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494858 2579 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494861 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494864 2579 flags.go:64] FLAG: --pods-per-core="0" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494866 2579 flags.go:64] FLAG: --port="10250" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494870 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494874 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ba840fad30329c80" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494877 2579 flags.go:64] FLAG: --qos-reserved="" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494882 2579 flags.go:64] FLAG: --read-only-port="10255" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494887 2579 flags.go:64] FLAG: --register-node="true" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494891 2579 flags.go:64] FLAG: --register-with-taints="" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494896 2579 flags.go:64] FLAG: --registry-burst="10" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494903 2579 flags.go:64] FLAG: --registry-qps="5" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494905 2579 flags.go:64] FLAG: --reserved-cpus="" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494908 2579 flags.go:64] FLAG: --reserved-memory="" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494912 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 06:32:04.804390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494915 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494918 2579 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494921 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494924 2579 flags.go:64] FLAG: --runonce="false" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494927 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494930 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494933 2579 flags.go:64] FLAG: --seccomp-default="false" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494935 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494938 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494941 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494944 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494947 2579 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494950 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494953 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494956 2579 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494959 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494961 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494964 2579 flags.go:64] FLAG: --system-cgroups="" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494967 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494972 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494975 2579 flags.go:64] FLAG: --tls-cert-file="" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494978 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494984 2579 flags.go:64] FLAG: --tls-min-version="" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494987 2579 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494989 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 06:32:04.806008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494992 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494995 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.494997 2579 flags.go:64] FLAG: --v="2" Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.495002 2579 flags.go:64] FLAG: --version="false" Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.495006 2579 flags.go:64] FLAG: --vmodule="" Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.495010 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.495013 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495137 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495140 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495143 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495146 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495149 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495151 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495154 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495157 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495159 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495162 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495164 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495167 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495169 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495172 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 06:32:04.807892 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495174 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495177 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495180 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495182 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495185 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495187 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495190 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495192 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495196 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495199 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495201 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495204 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495207 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495209 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495213 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495215 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495219 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495221 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495224 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495226 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 07 06:32:04.808782 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495228 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495231 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495234 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495236 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495239 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495241 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495244 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495246 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495249 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495251 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495253 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495256 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495259 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495261 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495264 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495266 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495268 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495271 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495274 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495276 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 06:32:04.809618 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495281 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495283 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495286 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495288 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495291 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495293 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495297 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495300 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495302 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495305 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495307 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495310 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495313 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495315 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495319 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495322 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495325 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495328 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495332 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 06:32:04.810429 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495335 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495338 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495341 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495344 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495347 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495350 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495352 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495355 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495357 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495360 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495376 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495378 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495381 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495385 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495387 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495390 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495392 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495395 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495397 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 06:32:04.811219 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495401 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 06:32:04.812004 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495405 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 06:32:04.812004 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495409 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 06:32:04.812004 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495413 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 06:32:04.812004 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495417 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 06:32:04.812004 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495420 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 06:32:04.812004 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.495423 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 06:32:04.812004 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.496505 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 06:32:04.812004 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.507830 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 06:32:04.812004 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.507848 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 06:32:04.812004 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.507867 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 06:32:04.812004 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.507873 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 06:32:04.812004 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507927 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 06:32:04.812004 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507934 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 06:32:04.812004 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507937 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507940 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507943 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507946 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507949 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507952 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507955 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507958 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507961 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507964 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507967 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507970 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507972 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507975 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507977 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507980 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507983 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507985 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507988 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 06:32:04.812596 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507990 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507993 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507996 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.507998 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508001 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508003 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508006 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508010 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508015 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508018 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508021 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508023 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508026 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508029 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508031 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508034 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508036 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508039 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508041 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 06:32:04.813337 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508044 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508046 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508049 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508051 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508054 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508056 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508059 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508061 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508064 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508067 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508069 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508072 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508074 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508076 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508079 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508082 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508085 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508087 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508090 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508092 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 06:32:04.814094 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508095 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508102 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508105 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508109 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508112 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508115 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508118 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508121 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508124 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508126 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508129 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508132 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508134 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508136 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508139 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508142 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508145 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508147 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508150 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508153 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 06:32:04.814904 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508155 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508158 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508160 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508163 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508165 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508168 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508171 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508174 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508176 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508179 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508181 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508184 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508186 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508190 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508193 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508196 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508199 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508201 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 06:32:04.815709 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508204 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 06:32:04.816477 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.508209 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 06:32:04.816477 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508315 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 06:32:04.816477 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508320 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 06:32:04.816477 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508323 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 06:32:04.816477 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508326 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 06:32:04.816477 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508329 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 06:32:04.816477 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508331 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 06:32:04.816477 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508334 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 06:32:04.816477 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508336 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 06:32:04.816477 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508339 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 06:32:04.816477 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508341 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 06:32:04.816477 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508344 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 06:32:04.816477 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508347 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 06:32:04.816477 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508349 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 06:32:04.816477 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508352 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508354 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508357 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508359 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508377 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508380 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508382 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508385 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508387 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508390 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508393 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508396 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508398 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508402 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508405 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508408 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508411 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508414 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508416 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 06:32:04.817073 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508419 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508422 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508424 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508427 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508429 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508432 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508435 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508437 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508441 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508445 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508447 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508450 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508453 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508455 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508458 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508461 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508463 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508466 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508471 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 06:32:04.817828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508473 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508476 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508478 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508481 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508484 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508486 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508489 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508493 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508496 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508498 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508501 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508504 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508507 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508509 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508511 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508514 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508517 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508519 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508522 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508524 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 06:32:04.818599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508527 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508529 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508532 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508534 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508537 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508539 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508541 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508544 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508547 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508549 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508552 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508555 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508558 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508560 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508563 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508565 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508569 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508573 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508576 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508579 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 06:32:04.819374 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508582 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 06:32:04.820183 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508585 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 06:32:04.820183 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508589 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 06:32:04.820183 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508592 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 06:32:04.820183 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508595 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 06:32:04.820183 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508598 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 06:32:04.820183 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508601 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 06:32:04.820183 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:04.508604 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 06:32:04.820183 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.508608 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 06:32:04.820183 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.508717 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 06:32:04.820183 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.510751 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 06:32:04.820183 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.512132 2579 server.go:1019] "Starting client certificate rotation" Jul 07 06:32:04.820183 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.512238 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 06:32:04.820183 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.512272 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 06:32:04.820732 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.514755 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 06:32:04.820732 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.515880 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 06:32:04.820732 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.529253 2579 log.go:25] "Validated CRI v1 runtime API" Jul 07 06:32:04.820732 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.531631 2579 log.go:25] "Validated CRI v1 image API" Jul 07 06:32:04.820732 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.532417 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 06:32:04.820732 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.534898 2579 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8a365d4c-aeaa-478d-972f-fed596645d49:/dev/nvme0n1p4 fa1d46a3-b117-451e-9591-682ce18d96e0:/dev/nvme0n1p3] Jul 07 06:32:04.820732 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.534914 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.539754 2579 manager.go:216] Machine: {Timestamp:2026-07-07 06:32:04.538683467 +0000 UTC m=+0.378524425 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099019 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2474c239e2264255cc98be1a01cdbc SystemUUID:ec2474c2-39e2-2642-55cc-98be1a01cdbc BootID:2ce54e8e-ba5b-4a97-bb94-280b697cb4e0 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:57:19:9e:8a:df Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:57:19:9e:8a:df Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:c7:18:21:a5:81 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.539845 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.539945 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.543445 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.543473 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-176.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.544275 2579 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.544287 2579 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.544298 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.545137 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.547125 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.550591 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.550730 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.551495 2579 kubelet.go:486] "Attempting to sync node with API server" Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.551508 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.551523 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.551534 2579 kubelet.go:398] "Adding apiserver pod source" Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.551545 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 06:32:04.821025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.551926 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.551945 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.554182 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.556591 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.556626 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.558153 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.558169 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.558177 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.558185 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.558197 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.558206 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.558218 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.558226 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.558235 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.558242 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.558261 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.558269 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.559571 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.559582 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.563207 2579 server.go:1268] "Started kubelet" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.563710 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.565412 2579 server.go:310] "Adding debug handlers to kubelet server" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.566311 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.566349 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.567621 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-176.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.567845 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-176.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.568057 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.569271 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.570281 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.570547 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v5qn5" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.574057 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.574201 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-176.ec2.internal.18bfee3d4fea691a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-176.ec2.internal,UID:ip-10-0-130-176.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-176.ec2.internal,},FirstTimestamp:2026-07-07 06:32:04.563183898 +0000 UTC m=+0.403024858,LastTimestamp:2026-07-07 06:32:04.563183898 +0000 UTC m=+0.403024858,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-176.ec2.internal,}" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.578251 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.578569 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.581027 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-176.ec2.internal\" not found" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.581668 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.581685 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.581880 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.581930 2579 reconstruct.go:97] "Volume reconstruction finished" Jul 07 06:32:04.821840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.581939 2579 reconciler.go:29] "Reconciler: start to sync state" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.582535 2579 factory.go:55] Registering systemd factory Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.582567 2579 factory.go:223] Registration of the systemd container factory successfully Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.582765 2579 factory.go:153] Registering CRI-O factory Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.582772 2579 factory.go:223] Registration of the crio container factory successfully Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.582812 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.582829 2579 factory.go:103] Registering Raw factory Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.582840 2579 manager.go:1196] Started watching for new ooms in manager Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.583115 2579 manager.go:349] Starting recovery of all containers Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.585474 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.589215 2579 manager.go:354] Recovery completed Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.596872 2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-176.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.596996 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.607076 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v5qn5" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.613106 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.614944 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasSufficientMemory" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.614971 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.614995 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasSufficientPID" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.615580 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.615589 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.615591 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.615608 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.616236 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasSufficientMemory" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.616258 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.616267 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasSufficientPID" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.616779 2579 policy_none.go:49] "None policy: Start" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.616793 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.616803 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.618273 2579 policy_none.go:47] "Start" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.668016 2579 manager.go:337] "Starting Device Plugin manager" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.668237 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.668263 2579 server.go:87] "Starting device plugin registration server" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.668523 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.668533 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.668654 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.669100 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.669113 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.670333 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.670409 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-176.ec2.internal\" not found" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.769659 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.770753 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasSufficientMemory" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.770785 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.770796 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasSufficientPID" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.770823 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-176.ec2.internal" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.776613 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.776652 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.776674 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 06:32:04.823408 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.776707 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 06:32:04.825301 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.778728 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 06:32:04.825301 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.780214 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-176.ec2.internal" Jul 07 06:32:04.825301 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.780233 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-176.ec2.internal\": node \"ip-10-0-130-176.ec2.internal\" not found" Jul 07 06:32:04.825301 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.812828 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-176.ec2.internal\" not found" Jul 07 06:32:04.880031 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.880005 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-176.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal"] Jul 07 06:32:04.880132 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.880085 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 06:32:04.881822 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.881805 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasSufficientMemory" Jul 07 06:32:04.881903 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.881834 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 06:32:04.881903 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.881846 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasSufficientPID" Jul 07 06:32:04.883046 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.883025 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b737b79e957a55e359ff156469d79a5e-config\") pod \"kube-apiserver-proxy-ip-10-0-130-176.ec2.internal\" (UID: \"b737b79e957a55e359ff156469d79a5e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-176.ec2.internal" Jul 07 06:32:04.883094 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.883078 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 06:32:04.883313 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.883263 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-176.ec2.internal" Jul 07 06:32:04.883313 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.883294 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 06:32:04.883831 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.883814 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasSufficientMemory" Jul 07 06:32:04.883906 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.883841 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 06:32:04.883906 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.883856 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasSufficientPID" Jul 07 06:32:04.883906 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.883893 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasSufficientMemory" Jul 07 06:32:04.884013 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.883913 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 06:32:04.884013 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.883923 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasSufficientPID" Jul 07 06:32:04.885939 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.885923 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal" Jul 07 06:32:04.886025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.885951 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 06:32:04.887835 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.887819 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasSufficientMemory" Jul 07 06:32:04.887912 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.887839 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 06:32:04.887912 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.887848 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeHasSufficientPID" Jul 07 06:32:04.908249 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.908226 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-176.ec2.internal\" not found" node="ip-10-0-130-176.ec2.internal" Jul 07 06:32:04.912843 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.912827 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-176.ec2.internal\" not found" node="ip-10-0-130-176.ec2.internal" Jul 07 06:32:04.913866 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:04.913851 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-176.ec2.internal\" not found" Jul 07 06:32:04.984083 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.984060 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b737b79e957a55e359ff156469d79a5e-config\") pod \"kube-apiserver-proxy-ip-10-0-130-176.ec2.internal\" (UID: \"b737b79e957a55e359ff156469d79a5e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-176.ec2.internal" Jul 07 06:32:04.984195 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.984098 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f47328e32847cb792862c0d80bc10faf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal\" (UID: \"f47328e32847cb792862c0d80bc10faf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal" Jul 07 06:32:04.984195 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.984121 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f47328e32847cb792862c0d80bc10faf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal\" (UID: \"f47328e32847cb792862c0d80bc10faf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal" Jul 07 06:32:04.984195 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:04.984182 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b737b79e957a55e359ff156469d79a5e-config\") pod \"kube-apiserver-proxy-ip-10-0-130-176.ec2.internal\" (UID: \"b737b79e957a55e359ff156469d79a5e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-176.ec2.internal" Jul 07 06:32:05.014851 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:05.014781 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-176.ec2.internal\" not found" Jul 07 06:32:05.084474 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.084443 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f47328e32847cb792862c0d80bc10faf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal\" (UID: \"f47328e32847cb792862c0d80bc10faf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal" Jul 07 06:32:05.084594 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.084478 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f47328e32847cb792862c0d80bc10faf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal\" (UID: \"f47328e32847cb792862c0d80bc10faf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal" Jul 07 06:32:05.084594 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.084569 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f47328e32847cb792862c0d80bc10faf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal\" (UID: \"f47328e32847cb792862c0d80bc10faf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal" Jul 07 06:32:05.084660 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.084621 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f47328e32847cb792862c0d80bc10faf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal\" (UID: \"f47328e32847cb792862c0d80bc10faf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal" Jul 07 06:32:05.116704 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:05.116658 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-176.ec2.internal\" not found" Jul 07 06:32:05.214463 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.214432 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-176.ec2.internal" Jul 07 06:32:05.217812 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:05.217778 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-176.ec2.internal\" not found" Jul 07 06:32:05.217812 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.217805 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal" Jul 07 06:32:05.319873 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:05.319789 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-176.ec2.internal\" not found" Jul 07 06:32:05.422034 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:05.422001 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-176.ec2.internal\" not found" Jul 07 06:32:05.511722 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.511696 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 06:32:05.512160 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.511859 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 06:32:05.522328 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:05.522301 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-176.ec2.internal\" not found" Jul 07 06:32:05.610967 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.610879 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 06:27:05 +0000 UTC" deadline="2028-01-15 03:05:10.674145168 +0000 UTC" Jul 07 06:32:05.610967 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.610948 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13364h33m5.063231061s" Jul 07 06:32:05.623125 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:05.623103 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-176.ec2.internal\" not found" Jul 07 06:32:05.662168 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.662145 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 06:32:05.723679 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:05.723640 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-176.ec2.internal\" not found" Jul 07 06:32:05.779858 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:05.779809 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb737b79e957a55e359ff156469d79a5e.slice/crio-9269f904782d6df423244368971918d10f425561545a17ddc3a378b6c351ec48 WatchSource:0}: Error finding container 9269f904782d6df423244368971918d10f425561545a17ddc3a378b6c351ec48: Status 404 returned error can't find the container with id 9269f904782d6df423244368971918d10f425561545a17ddc3a378b6c351ec48 Jul 07 06:32:05.780586 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:05.780566 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf47328e32847cb792862c0d80bc10faf.slice/crio-27496462100fc18aefb30ee80b336b322802dc21287b47c209ee393da5334767 WatchSource:0}: Error finding container 27496462100fc18aefb30ee80b336b322802dc21287b47c209ee393da5334767: Status 404 returned error can't find the container with id 27496462100fc18aefb30ee80b336b322802dc21287b47c209ee393da5334767 Jul 07 06:32:05.787815 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.787794 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 06:32:05.824047 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:05.824012 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-176.ec2.internal\" not found" Jul 07 06:32:05.926151 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:05.926076 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-176.ec2.internal\" not found" Jul 07 06:32:05.959195 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.959170 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 06:32:05.982719 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.982695 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-176.ec2.internal" Jul 07 06:32:05.990264 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.990238 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 06:32:05.992104 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.992090 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 06:32:05.993131 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:05.993116 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal" Jul 07 06:32:06.005721 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.005705 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 06:32:06.334135 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.334045 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 06:32:06.552908 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.552880 2579 apiserver.go:52] "Watching apiserver" Jul 07 06:32:06.564799 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.563744 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 06:32:06.564799 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.564414 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-mqsn2","openshift-network-operator/iptables-alerter-vkw7h","openshift-ovn-kubernetes/ovnkube-node-vn8hx","kube-system/global-pull-secret-syncer-4bdk4","kube-system/konnectivity-agent-5q8w9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx","openshift-cluster-node-tuning-operator/tuned-8kknw","openshift-image-registry/node-ca-zjmvc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal","openshift-multus/multus-additional-cni-plugins-f9mqv","openshift-multus/network-metrics-daemon-thndk","kube-system/kube-apiserver-proxy-ip-10-0-130-176.ec2.internal","openshift-dns/node-resolver-86q7n","openshift-multus/multus-j6ngv"] Jul 07 06:32:06.566790 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.566178 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.567733 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.567710 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-86q7n" Jul 07 06:32:06.569429 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.569115 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.570741 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.570672 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 06:32:06.570861 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.570784 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 06:32:06.570861 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.570682 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xz9j9\"" Jul 07 06:32:06.571165 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.571139 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:06.571289 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:06.571244 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:06.571361 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.571312 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 06:32:06.571601 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.571574 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4tg7h\"" Jul 07 06:32:06.574871 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.573180 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 06:32:06.574871 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.573654 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5q8w9" Jul 07 06:32:06.575155 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.575133 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 06:32:06.575680 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.575656 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 06:32:06.576122 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.576093 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 06:32:06.576558 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.576525 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 06:32:06.576993 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.576970 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 06:32:06.577951 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.577636 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 06:32:06.577951 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.577679 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-m97kb\"" Jul 07 06:32:06.577951 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.577738 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.580559 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.580537 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-k6plg\"" Jul 07 06:32:06.580657 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.580545 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 06:32:06.582024 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.581079 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 06:32:06.582024 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.581285 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 06:32:06.582024 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.581438 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vxfsc\"" Jul 07 06:32:06.582024 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.581616 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 06:32:06.582024 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.581864 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 06:32:06.582024 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.581913 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 06:32:06.582363 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.582273 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.583891 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.583532 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:06.583891 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:06.583597 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:06.585119 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.585049 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 06:32:06.585304 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.585285 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 06:32:06.585640 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.585620 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 06:32:06.585640 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.585620 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 06:32:06.585783 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.585629 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vkw7h" Jul 07 06:32:06.586248 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.586229 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6rq4j\"" Jul 07 06:32:06.586585 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.586566 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.587859 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.587839 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:06.587969 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:06.587919 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:06.588263 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.588243 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pr94r\"" Jul 07 06:32:06.588716 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.588699 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 06:32:06.589135 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.589116 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 06:32:06.589358 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.589338 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zjmvc" Jul 07 06:32:06.589535 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.589517 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 06:32:06.589754 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.589736 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 06:32:06.590182 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.590062 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 06:32:06.590182 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.590074 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9zldw\"" Jul 07 06:32:06.593120 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.592892 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 06:32:06.593120 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.592946 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 06:32:06.593120 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.592947 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 06:32:06.593120 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.592892 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jj4sd\"" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.595905 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/606bdf3e-ea47-4a60-a2fd-cf5aa63d341c-konnectivity-ca\") pod \"konnectivity-agent-5q8w9\" (UID: \"606bdf3e-ea47-4a60-a2fd-cf5aa63d341c\") " pod="kube-system/konnectivity-agent-5q8w9" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.595982 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-socket-dir\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596011 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-run-multus-certs\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596059 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdgbq\" (UniqueName: \"kubernetes.io/projected/228e7186-241d-4d93-9874-ee588f0a5e1d-kube-api-access-xdgbq\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596086 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e778e873-cfa7-4f30-9190-d2cfa231a778-tmp\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596105 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596122 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcd8d\" (UniqueName: \"kubernetes.io/projected/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-kube-api-access-bcd8d\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596140 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-var-lib-kubelet\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596156 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-sysctl-conf\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596171 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8deda72-2465-458f-af82-23eb7bd4156f-tmp-dir\") pod \"node-resolver-86q7n\" (UID: \"d8deda72-2465-458f-af82-23eb7bd4156f\") " pod="openshift-dns/node-resolver-86q7n" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596188 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-registration-dir\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596206 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-cnibin\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596219 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-hostroot\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596234 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-multus-conf-dir\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596248 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m72px\" (UniqueName: \"kubernetes.io/projected/e778e873-cfa7-4f30-9190-d2cfa231a778-kube-api-access-m72px\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.596563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596276 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-systemd-units\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596318 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/997fb304-b3ae-42e0-8a84-db42daea16b1-dbus\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596356 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/228e7186-241d-4d93-9874-ee588f0a5e1d-cni-binary-copy\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596388 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-run\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596441 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-run-ovn-kubernetes\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596478 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-system-cni-dir\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596519 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4bf5\" (UniqueName: \"kubernetes.io/projected/d8deda72-2465-458f-af82-23eb7bd4156f-kube-api-access-c4bf5\") pod \"node-resolver-86q7n\" (UID: \"d8deda72-2465-458f-af82-23eb7bd4156f\") " pod="openshift-dns/node-resolver-86q7n" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596543 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-kubelet\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596588 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/228e7186-241d-4d93-9874-ee588f0a5e1d-multus-daemon-config\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596617 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-modprobe-d\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596640 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-var-lib-kubelet\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596673 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-host\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596739 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-tuned\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596773 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596800 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-systemd\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596851 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-slash\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596880 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-run-systemd\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.597378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596936 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8deda72-2465-458f-af82-23eb7bd4156f-hosts-file\") pod \"node-resolver-86q7n\" (UID: \"d8deda72-2465-458f-af82-23eb7bd4156f\") " pod="openshift-dns/node-resolver-86q7n" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596963 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a328cd98-578d-4f2a-a622-0739620a9a6f-ovnkube-config\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.596988 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a328cd98-578d-4f2a-a622-0739620a9a6f-env-overrides\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.597023 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-run-netns\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.597091 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-var-lib-cni-bin\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.597119 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/997fb304-b3ae-42e0-8a84-db42daea16b1-kubelet-config\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.597145 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.597169 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-multus-cni-dir\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.597255 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsp8h\" (UniqueName: \"kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h\") pod \"network-check-target-mqsn2\" (UID: \"d592a475-9102-419a-a6e0-5877d1838ca9\") " pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.597297 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-lib-modules\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.597321 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-run-ovn\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.597345 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-os-release\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.597384 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-multus-socket-dir-parent\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.597408 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-sys\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.597439 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-etc-openvswitch\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.597466 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-run-openvswitch\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.598338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.597489 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-node-log\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.599225 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.597515 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.599814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599389 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-var-lib-cni-multus\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.599814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599457 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-cni-netd\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.599814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599496 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-kubernetes\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.599814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599538 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-sysctl-d\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.599814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599564 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-log-socket\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.599814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599595 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a328cd98-578d-4f2a-a622-0739620a9a6f-ovnkube-script-lib\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.599814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599633 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/606bdf3e-ea47-4a60-a2fd-cf5aa63d341c-agent-certs\") pod \"konnectivity-agent-5q8w9\" (UID: \"606bdf3e-ea47-4a60-a2fd-cf5aa63d341c\") " pod="kube-system/konnectivity-agent-5q8w9" Jul 07 06:32:06.599814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599661 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-cni-bin\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.599814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599670 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 06:32:06.599814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599694 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x47wd\" (UniqueName: \"kubernetes.io/projected/a328cd98-578d-4f2a-a622-0739620a9a6f-kube-api-access-x47wd\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.599814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599723 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-sys-fs\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.599814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599750 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-run-k8s-cni-cncf-io\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.599814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599785 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-run-netns\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.600670 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599837 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-etc-kubernetes\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.600670 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599859 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-sysconfig\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.600670 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599927 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-var-lib-openvswitch\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.600670 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599967 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a328cd98-578d-4f2a-a622-0739620a9a6f-ovn-node-metrics-cert\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.600670 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.599998 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-device-dir\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.618523 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.618493 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5dkg8" Jul 07 06:32:06.630495 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.630027 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5dkg8" Jul 07 06:32:06.683894 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.683801 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 06:32:06.700357 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700327 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a328cd98-578d-4f2a-a622-0739620a9a6f-ovn-node-metrics-cert\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.700493 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700376 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/acb18573-2d7c-4e7b-9429-dcad9be2ced6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.700493 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700424 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-device-dir\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.700493 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700450 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/606bdf3e-ea47-4a60-a2fd-cf5aa63d341c-konnectivity-ca\") pod \"konnectivity-agent-5q8w9\" (UID: \"606bdf3e-ea47-4a60-a2fd-cf5aa63d341c\") " pod="kube-system/konnectivity-agent-5q8w9" Jul 07 06:32:06.700493 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700474 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-socket-dir\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.700733 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700499 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-run-multus-certs\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.700733 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700524 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdgbq\" (UniqueName: \"kubernetes.io/projected/228e7186-241d-4d93-9874-ee588f0a5e1d-kube-api-access-xdgbq\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.700733 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700549 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e778e873-cfa7-4f30-9190-d2cfa231a778-tmp\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.700733 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700574 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca16c608-dfd6-48a9-84c6-6a35f862c77a-host-slash\") pod \"iptables-alerter-vkw7h\" (UID: \"ca16c608-dfd6-48a9-84c6-6a35f862c77a\") " pod="openshift-network-operator/iptables-alerter-vkw7h" Jul 07 06:32:06.700733 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700601 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acb18573-2d7c-4e7b-9429-dcad9be2ced6-system-cni-dir\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.700733 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700627 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.700733 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700654 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcd8d\" (UniqueName: \"kubernetes.io/projected/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-kube-api-access-bcd8d\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.700733 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700690 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-var-lib-kubelet\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.700733 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700715 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-sysctl-conf\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.701199 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700744 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8deda72-2465-458f-af82-23eb7bd4156f-tmp-dir\") pod \"node-resolver-86q7n\" (UID: \"d8deda72-2465-458f-af82-23eb7bd4156f\") " pod="openshift-dns/node-resolver-86q7n" Jul 07 06:32:06.701199 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700771 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/acb18573-2d7c-4e7b-9429-dcad9be2ced6-cni-binary-copy\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.701199 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700798 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-registration-dir\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.701199 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700800 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 06:32:06.701199 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.700880 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-registration-dir\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.701199 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.701046 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.701524 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.701439 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-sysctl-conf\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.701807 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.701780 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-run-multus-certs\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.701807 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.701801 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8deda72-2465-458f-af82-23eb7bd4156f-tmp-dir\") pod \"node-resolver-86q7n\" (UID: \"d8deda72-2465-458f-af82-23eb7bd4156f\") " pod="openshift-dns/node-resolver-86q7n" Jul 07 06:32:06.701955 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.701912 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-socket-dir\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.701955 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.701938 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/606bdf3e-ea47-4a60-a2fd-cf5aa63d341c-konnectivity-ca\") pod \"konnectivity-agent-5q8w9\" (UID: \"606bdf3e-ea47-4a60-a2fd-cf5aa63d341c\") " pod="kube-system/konnectivity-agent-5q8w9" Jul 07 06:32:06.702108 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.701978 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-device-dir\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.702108 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702015 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-var-lib-kubelet\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.702219 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702110 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-cnibin\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.702219 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702149 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-hostroot\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.702219 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702175 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-multus-conf-dir\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.702219 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702205 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-hostroot\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.702219 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702205 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m72px\" (UniqueName: \"kubernetes.io/projected/e778e873-cfa7-4f30-9190-d2cfa231a778-kube-api-access-m72px\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.702491 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702221 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-cnibin\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.702491 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702254 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-systemd-units\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.702491 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702270 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-multus-conf-dir\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.702491 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702285 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ca16c608-dfd6-48a9-84c6-6a35f862c77a-iptables-alerter-script\") pod \"iptables-alerter-vkw7h\" (UID: \"ca16c608-dfd6-48a9-84c6-6a35f862c77a\") " pod="openshift-network-operator/iptables-alerter-vkw7h" Jul 07 06:32:06.702491 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702311 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-systemd-units\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.702491 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702321 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/997fb304-b3ae-42e0-8a84-db42daea16b1-dbus\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:06.702491 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702351 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/228e7186-241d-4d93-9874-ee588f0a5e1d-cni-binary-copy\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.702491 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702461 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-run\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.702491 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702490 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-run-ovn-kubernetes\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702518 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs\") pod \"network-metrics-daemon-thndk\" (UID: \"e3255519-1e37-4781-8d6e-14394758a8b2\") " pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702546 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-system-cni-dir\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702571 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39868a40-e8ab-4965-b28a-83e1a6a46162-host\") pod \"node-ca-zjmvc\" (UID: \"39868a40-e8ab-4965-b28a-83e1a6a46162\") " pod="openshift-image-registry/node-ca-zjmvc" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702582 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/997fb304-b3ae-42e0-8a84-db42daea16b1-dbus\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702596 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzqrx\" (UniqueName: \"kubernetes.io/projected/39868a40-e8ab-4965-b28a-83e1a6a46162-kube-api-access-dzqrx\") pod \"node-ca-zjmvc\" (UID: \"39868a40-e8ab-4965-b28a-83e1a6a46162\") " pod="openshift-image-registry/node-ca-zjmvc" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702625 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4bf5\" (UniqueName: \"kubernetes.io/projected/d8deda72-2465-458f-af82-23eb7bd4156f-kube-api-access-c4bf5\") pod \"node-resolver-86q7n\" (UID: \"d8deda72-2465-458f-af82-23eb7bd4156f\") " pod="openshift-dns/node-resolver-86q7n" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702651 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-run\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702652 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-kubelet\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702694 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/228e7186-241d-4d93-9874-ee588f0a5e1d-multus-daemon-config\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702722 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-modprobe-d\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702741 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-system-cni-dir\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702751 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-var-lib-kubelet\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702695 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-kubelet\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702777 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-host\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702843 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-run-ovn-kubernetes\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702888 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-modprobe-d\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702891 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/228e7186-241d-4d93-9874-ee588f0a5e1d-cni-binary-copy\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.703008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702919 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-tuned\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.702973 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703000 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-systemd\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703056 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-slash\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703082 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-run-systemd\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703106 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8deda72-2465-458f-af82-23eb7bd4156f-hosts-file\") pod \"node-resolver-86q7n\" (UID: \"d8deda72-2465-458f-af82-23eb7bd4156f\") " pod="openshift-dns/node-resolver-86q7n" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703154 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a328cd98-578d-4f2a-a622-0739620a9a6f-ovnkube-config\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703200 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a328cd98-578d-4f2a-a622-0739620a9a6f-env-overrides\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703234 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/acb18573-2d7c-4e7b-9429-dcad9be2ced6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703264 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-run-netns\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703291 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-var-lib-cni-bin\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703317 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/997fb304-b3ae-42e0-8a84-db42daea16b1-kubelet-config\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703372 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703389 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/228e7186-241d-4d93-9874-ee588f0a5e1d-multus-daemon-config\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703397 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-multus-cni-dir\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703425 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsp8h\" (UniqueName: \"kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h\") pod \"network-check-target-mqsn2\" (UID: \"d592a475-9102-419a-a6e0-5877d1838ca9\") " pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703474 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-lib-modules\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.703932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703499 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-run-ovn\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703524 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-os-release\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703572 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-multus-socket-dir-parent\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703611 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-var-lib-kubelet\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703525 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-systemd\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703668 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-sys\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703695 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-host\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703699 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-etc-openvswitch\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703727 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-run-openvswitch\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703735 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703748 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8deda72-2465-458f-af82-23eb7bd4156f-hosts-file\") pod \"node-resolver-86q7n\" (UID: \"d8deda72-2465-458f-af82-23eb7bd4156f\") " pod="openshift-dns/node-resolver-86q7n" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:06.703754 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703794 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-multus-socket-dir-parent\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:06.703823 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret podName:997fb304-b3ae-42e0-8a84-db42daea16b1 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:07.203802016 +0000 UTC m=+3.043642986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret") pod "global-pull-secret-syncer-4bdk4" (UID: "997fb304-b3ae-42e0-8a84-db42daea16b1") : object "kube-system"/"original-pull-secret" not registered Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703816 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-multus-cni-dir\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703754 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-node-log\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703834 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a328cd98-578d-4f2a-a622-0739620a9a6f-env-overrides\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703852 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-slash\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.704852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703860 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-os-release\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703795 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-run-systemd\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703910 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-node-log\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703963 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.703970 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-etc-openvswitch\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704001 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/acb18573-2d7c-4e7b-9429-dcad9be2ced6-cnibin\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704009 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-run-netns\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704031 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-var-lib-cni-multus\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704048 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-run-ovn\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704058 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-cni-netd\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704081 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-sys\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704090 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntwf5\" (UniqueName: \"kubernetes.io/projected/acb18573-2d7c-4e7b-9429-dcad9be2ced6-kube-api-access-ntwf5\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704099 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-var-lib-cni-multus\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704096 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-var-lib-cni-bin\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704134 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-run-openvswitch\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704138 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/997fb304-b3ae-42e0-8a84-db42daea16b1-kubelet-config\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704161 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.705873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704180 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-lib-modules\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704229 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-kubernetes\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704241 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-cni-netd\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704284 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-sysctl-d\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704291 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-kubernetes\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704318 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-log-socket\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704354 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-log-socket\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704355 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a328cd98-578d-4f2a-a622-0739620a9a6f-ovnkube-script-lib\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704387 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-sysctl-d\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704402 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/acb18573-2d7c-4e7b-9429-dcad9be2ced6-os-release\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704441 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl6b9\" (UniqueName: \"kubernetes.io/projected/e3255519-1e37-4781-8d6e-14394758a8b2-kube-api-access-nl6b9\") pod \"network-metrics-daemon-thndk\" (UID: \"e3255519-1e37-4781-8d6e-14394758a8b2\") " pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704479 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/606bdf3e-ea47-4a60-a2fd-cf5aa63d341c-agent-certs\") pod \"konnectivity-agent-5q8w9\" (UID: \"606bdf3e-ea47-4a60-a2fd-cf5aa63d341c\") " pod="kube-system/konnectivity-agent-5q8w9" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704512 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-cni-bin\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704549 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-host-cni-bin\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704602 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x47wd\" (UniqueName: \"kubernetes.io/projected/a328cd98-578d-4f2a-a622-0739620a9a6f-kube-api-access-x47wd\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704851 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a328cd98-578d-4f2a-a622-0739620a9a6f-ovnkube-config\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704954 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4gpb\" (UniqueName: \"kubernetes.io/projected/ca16c608-dfd6-48a9-84c6-6a35f862c77a-kube-api-access-p4gpb\") pod \"iptables-alerter-vkw7h\" (UID: \"ca16c608-dfd6-48a9-84c6-6a35f862c77a\") " pod="openshift-network-operator/iptables-alerter-vkw7h" Jul 07 06:32:06.706767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704974 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a328cd98-578d-4f2a-a622-0739620a9a6f-ovnkube-script-lib\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.704998 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/acb18573-2d7c-4e7b-9429-dcad9be2ced6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.705024 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/39868a40-e8ab-4965-b28a-83e1a6a46162-serviceca\") pod \"node-ca-zjmvc\" (UID: \"39868a40-e8ab-4965-b28a-83e1a6a46162\") " pod="openshift-image-registry/node-ca-zjmvc" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.705079 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-sys-fs\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.705106 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-run-k8s-cni-cncf-io\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.705131 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-run-netns\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.705157 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-sys-fs\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.705167 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-etc-kubernetes\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.705201 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-run-netns\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.705203 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-sysconfig\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.705217 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-etc-kubernetes\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.705161 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/228e7186-241d-4d93-9874-ee588f0a5e1d-host-run-k8s-cni-cncf-io\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.705253 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-var-lib-openvswitch\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.705269 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-sysconfig\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.705368 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a328cd98-578d-4f2a-a622-0739620a9a6f-var-lib-openvswitch\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.707086 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e778e873-cfa7-4f30-9190-d2cfa231a778-tmp\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.707742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.707529 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a328cd98-578d-4f2a-a622-0739620a9a6f-ovn-node-metrics-cert\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.708561 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.708004 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/606bdf3e-ea47-4a60-a2fd-cf5aa63d341c-agent-certs\") pod \"konnectivity-agent-5q8w9\" (UID: \"606bdf3e-ea47-4a60-a2fd-cf5aa63d341c\") " pod="kube-system/konnectivity-agent-5q8w9" Jul 07 06:32:06.708767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.708746 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e778e873-cfa7-4f30-9190-d2cfa231a778-etc-tuned\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.715585 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.715508 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcd8d\" (UniqueName: \"kubernetes.io/projected/b25f9d15-2923-4c0a-a8dc-c7941ce430a0-kube-api-access-bcd8d\") pod \"aws-ebs-csi-driver-node-srfjx\" (UID: \"b25f9d15-2923-4c0a-a8dc-c7941ce430a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.715804 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:06.715785 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 06:32:06.715882 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:06.715812 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 06:32:06.715882 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:06.715827 2579 projected.go:196] Error preparing data for projected volume kube-api-access-vsp8h for pod openshift-network-diagnostics/network-check-target-mqsn2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:32:06.715991 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:06.715883 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h podName:d592a475-9102-419a-a6e0-5877d1838ca9 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:07.215864497 +0000 UTC m=+3.055705471 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vsp8h" (UniqueName: "kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h") pod "network-check-target-mqsn2" (UID: "d592a475-9102-419a-a6e0-5877d1838ca9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:32:06.719314 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.719286 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m72px\" (UniqueName: \"kubernetes.io/projected/e778e873-cfa7-4f30-9190-d2cfa231a778-kube-api-access-m72px\") pod \"tuned-8kknw\" (UID: \"e778e873-cfa7-4f30-9190-d2cfa231a778\") " pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.719540 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.719519 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4bf5\" (UniqueName: \"kubernetes.io/projected/d8deda72-2465-458f-af82-23eb7bd4156f-kube-api-access-c4bf5\") pod \"node-resolver-86q7n\" (UID: \"d8deda72-2465-458f-af82-23eb7bd4156f\") " pod="openshift-dns/node-resolver-86q7n" Jul 07 06:32:06.720661 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.720643 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdgbq\" (UniqueName: \"kubernetes.io/projected/228e7186-241d-4d93-9874-ee588f0a5e1d-kube-api-access-xdgbq\") pod \"multus-j6ngv\" (UID: \"228e7186-241d-4d93-9874-ee588f0a5e1d\") " pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.721867 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.721822 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x47wd\" (UniqueName: \"kubernetes.io/projected/a328cd98-578d-4f2a-a622-0739620a9a6f-kube-api-access-x47wd\") pod \"ovnkube-node-vn8hx\" (UID: \"a328cd98-578d-4f2a-a622-0739620a9a6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.782096 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.782045 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-176.ec2.internal" event={"ID":"b737b79e957a55e359ff156469d79a5e","Type":"ContainerStarted","Data":"9269f904782d6df423244368971918d10f425561545a17ddc3a378b6c351ec48"} Jul 07 06:32:06.783454 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.783407 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal" event={"ID":"f47328e32847cb792862c0d80bc10faf","Type":"ContainerStarted","Data":"27496462100fc18aefb30ee80b336b322802dc21287b47c209ee393da5334767"} Jul 07 06:32:06.806055 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806025 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4gpb\" (UniqueName: \"kubernetes.io/projected/ca16c608-dfd6-48a9-84c6-6a35f862c77a-kube-api-access-p4gpb\") pod \"iptables-alerter-vkw7h\" (UID: \"ca16c608-dfd6-48a9-84c6-6a35f862c77a\") " pod="openshift-network-operator/iptables-alerter-vkw7h" Jul 07 06:32:06.806202 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806068 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/acb18573-2d7c-4e7b-9429-dcad9be2ced6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.806202 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806096 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/39868a40-e8ab-4965-b28a-83e1a6a46162-serviceca\") pod \"node-ca-zjmvc\" (UID: \"39868a40-e8ab-4965-b28a-83e1a6a46162\") " pod="openshift-image-registry/node-ca-zjmvc" Jul 07 06:32:06.806202 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806160 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/acb18573-2d7c-4e7b-9429-dcad9be2ced6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.806346 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806212 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca16c608-dfd6-48a9-84c6-6a35f862c77a-host-slash\") pod \"iptables-alerter-vkw7h\" (UID: \"ca16c608-dfd6-48a9-84c6-6a35f862c77a\") " pod="openshift-network-operator/iptables-alerter-vkw7h" Jul 07 06:32:06.806346 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806241 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acb18573-2d7c-4e7b-9429-dcad9be2ced6-system-cni-dir\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.806346 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806272 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/acb18573-2d7c-4e7b-9429-dcad9be2ced6-cni-binary-copy\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.806346 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806305 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ca16c608-dfd6-48a9-84c6-6a35f862c77a-iptables-alerter-script\") pod \"iptables-alerter-vkw7h\" (UID: \"ca16c608-dfd6-48a9-84c6-6a35f862c77a\") " pod="openshift-network-operator/iptables-alerter-vkw7h" Jul 07 06:32:06.806530 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806334 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs\") pod \"network-metrics-daemon-thndk\" (UID: \"e3255519-1e37-4781-8d6e-14394758a8b2\") " pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:06.806530 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806371 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39868a40-e8ab-4965-b28a-83e1a6a46162-host\") pod \"node-ca-zjmvc\" (UID: \"39868a40-e8ab-4965-b28a-83e1a6a46162\") " pod="openshift-image-registry/node-ca-zjmvc" Jul 07 06:32:06.806530 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806395 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzqrx\" (UniqueName: \"kubernetes.io/projected/39868a40-e8ab-4965-b28a-83e1a6a46162-kube-api-access-dzqrx\") pod \"node-ca-zjmvc\" (UID: \"39868a40-e8ab-4965-b28a-83e1a6a46162\") " pod="openshift-image-registry/node-ca-zjmvc" Jul 07 06:32:06.806530 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806414 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acb18573-2d7c-4e7b-9429-dcad9be2ced6-system-cni-dir\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.806530 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806479 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/acb18573-2d7c-4e7b-9429-dcad9be2ced6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.806820 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806537 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/acb18573-2d7c-4e7b-9429-dcad9be2ced6-cnibin\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.806820 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806567 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntwf5\" (UniqueName: \"kubernetes.io/projected/acb18573-2d7c-4e7b-9429-dcad9be2ced6-kube-api-access-ntwf5\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.806820 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806574 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/39868a40-e8ab-4965-b28a-83e1a6a46162-serviceca\") pod \"node-ca-zjmvc\" (UID: \"39868a40-e8ab-4965-b28a-83e1a6a46162\") " pod="openshift-image-registry/node-ca-zjmvc" Jul 07 06:32:06.806820 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806600 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/acb18573-2d7c-4e7b-9429-dcad9be2ced6-os-release\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.806820 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806676 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/acb18573-2d7c-4e7b-9429-dcad9be2ced6-os-release\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.806820 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806720 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/acb18573-2d7c-4e7b-9429-dcad9be2ced6-cnibin\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.806820 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806769 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/acb18573-2d7c-4e7b-9429-dcad9be2ced6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.807183 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:06.806821 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:32:06.807183 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806829 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca16c608-dfd6-48a9-84c6-6a35f862c77a-host-slash\") pod \"iptables-alerter-vkw7h\" (UID: \"ca16c608-dfd6-48a9-84c6-6a35f862c77a\") " pod="openshift-network-operator/iptables-alerter-vkw7h" Jul 07 06:32:06.807183 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.806880 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39868a40-e8ab-4965-b28a-83e1a6a46162-host\") pod \"node-ca-zjmvc\" (UID: \"39868a40-e8ab-4965-b28a-83e1a6a46162\") " pod="openshift-image-registry/node-ca-zjmvc" Jul 07 06:32:06.807183 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:06.806888 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs podName:e3255519-1e37-4781-8d6e-14394758a8b2 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:07.306869473 +0000 UTC m=+3.146710423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs") pod "network-metrics-daemon-thndk" (UID: "e3255519-1e37-4781-8d6e-14394758a8b2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:32:06.807183 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.807008 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl6b9\" (UniqueName: \"kubernetes.io/projected/e3255519-1e37-4781-8d6e-14394758a8b2-kube-api-access-nl6b9\") pod \"network-metrics-daemon-thndk\" (UID: \"e3255519-1e37-4781-8d6e-14394758a8b2\") " pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:06.807183 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.807179 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/acb18573-2d7c-4e7b-9429-dcad9be2ced6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.807540 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.807344 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/acb18573-2d7c-4e7b-9429-dcad9be2ced6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.807540 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.807377 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ca16c608-dfd6-48a9-84c6-6a35f862c77a-iptables-alerter-script\") pod \"iptables-alerter-vkw7h\" (UID: \"ca16c608-dfd6-48a9-84c6-6a35f862c77a\") " pod="openshift-network-operator/iptables-alerter-vkw7h" Jul 07 06:32:06.808401 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.808357 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/acb18573-2d7c-4e7b-9429-dcad9be2ced6-cni-binary-copy\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.821484 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.821415 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzqrx\" (UniqueName: \"kubernetes.io/projected/39868a40-e8ab-4965-b28a-83e1a6a46162-kube-api-access-dzqrx\") pod \"node-ca-zjmvc\" (UID: \"39868a40-e8ab-4965-b28a-83e1a6a46162\") " pod="openshift-image-registry/node-ca-zjmvc" Jul 07 06:32:06.822318 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.821786 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4gpb\" (UniqueName: \"kubernetes.io/projected/ca16c608-dfd6-48a9-84c6-6a35f862c77a-kube-api-access-p4gpb\") pod \"iptables-alerter-vkw7h\" (UID: \"ca16c608-dfd6-48a9-84c6-6a35f862c77a\") " pod="openshift-network-operator/iptables-alerter-vkw7h" Jul 07 06:32:06.822318 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.822226 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntwf5\" (UniqueName: \"kubernetes.io/projected/acb18573-2d7c-4e7b-9429-dcad9be2ced6-kube-api-access-ntwf5\") pod \"multus-additional-cni-plugins-f9mqv\" (UID: \"acb18573-2d7c-4e7b-9429-dcad9be2ced6\") " pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.822318 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.822279 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl6b9\" (UniqueName: \"kubernetes.io/projected/e3255519-1e37-4781-8d6e-14394758a8b2-kube-api-access-nl6b9\") pod \"network-metrics-daemon-thndk\" (UID: \"e3255519-1e37-4781-8d6e-14394758a8b2\") " pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:06.885714 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.885637 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8kknw" Jul 07 06:32:06.894828 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:06.894788 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode778e873_cfa7_4f30_9190_d2cfa231a778.slice/crio-cc99ba9dde7241a2aee8ec8fa480df805768aa019210710afe94c4bfc47f0f70 WatchSource:0}: Error finding container cc99ba9dde7241a2aee8ec8fa480df805768aa019210710afe94c4bfc47f0f70: Status 404 returned error can't find the container with id cc99ba9dde7241a2aee8ec8fa480df805768aa019210710afe94c4bfc47f0f70 Jul 07 06:32:06.901711 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.901688 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5q8w9" Jul 07 06:32:06.908603 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.908583 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-86q7n" Jul 07 06:32:06.910797 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:06.910764 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod606bdf3e_ea47_4a60_a2fd_cf5aa63d341c.slice/crio-80c918b53e332b7a55c67fcd9a0dd3a77265cb14ab56d9e6093b1c5966b4d5f9 WatchSource:0}: Error finding container 80c918b53e332b7a55c67fcd9a0dd3a77265cb14ab56d9e6093b1c5966b4d5f9: Status 404 returned error can't find the container with id 80c918b53e332b7a55c67fcd9a0dd3a77265cb14ab56d9e6093b1c5966b4d5f9 Jul 07 06:32:06.915139 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.915113 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:06.919037 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:06.919015 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8deda72_2465_458f_af82_23eb7bd4156f.slice/crio-f3bae7c369235d1e17ea074a272a8c30a7b63a14ee72ac1c615aed7b14ad9a3a WatchSource:0}: Error finding container f3bae7c369235d1e17ea074a272a8c30a7b63a14ee72ac1c615aed7b14ad9a3a: Status 404 returned error can't find the container with id f3bae7c369235d1e17ea074a272a8c30a7b63a14ee72ac1c615aed7b14ad9a3a Jul 07 06:32:06.923152 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.923126 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" Jul 07 06:32:06.925172 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:06.925147 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda328cd98_578d_4f2a_a622_0739620a9a6f.slice/crio-a0b20da210b91b7b09cf77393483421a5e4942c5c50fecc5e3511423f2b1faed WatchSource:0}: Error finding container a0b20da210b91b7b09cf77393483421a5e4942c5c50fecc5e3511423f2b1faed: Status 404 returned error can't find the container with id a0b20da210b91b7b09cf77393483421a5e4942c5c50fecc5e3511423f2b1faed Jul 07 06:32:06.934542 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:06.934395 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25f9d15_2923_4c0a_a8dc_c7941ce430a0.slice/crio-931065cdaebec02d2c931f4c3c03347c2513e3eb2d02f0888f964524ae55d058 WatchSource:0}: Error finding container 931065cdaebec02d2c931f4c3c03347c2513e3eb2d02f0888f964524ae55d058: Status 404 returned error can't find the container with id 931065cdaebec02d2c931f4c3c03347c2513e3eb2d02f0888f964524ae55d058 Jul 07 06:32:06.935980 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.935957 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j6ngv" Jul 07 06:32:06.944689 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:06.944668 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod228e7186_241d_4d93_9874_ee588f0a5e1d.slice/crio-9728808c2b40030af233d0ef8f0c9ca298c060a8c6c670a76449a420ad8719f3 WatchSource:0}: Error finding container 9728808c2b40030af233d0ef8f0c9ca298c060a8c6c670a76449a420ad8719f3: Status 404 returned error can't find the container with id 9728808c2b40030af233d0ef8f0c9ca298c060a8c6c670a76449a420ad8719f3 Jul 07 06:32:06.946160 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.946139 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vkw7h" Jul 07 06:32:06.952322 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.952239 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f9mqv" Jul 07 06:32:06.954846 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:06.954821 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca16c608_dfd6_48a9_84c6_6a35f862c77a.slice/crio-10005e8cbc8b740386532359d0d89614d6d8d1902de2bd86494128d3e317713f WatchSource:0}: Error finding container 10005e8cbc8b740386532359d0d89614d6d8d1902de2bd86494128d3e317713f: Status 404 returned error can't find the container with id 10005e8cbc8b740386532359d0d89614d6d8d1902de2bd86494128d3e317713f Jul 07 06:32:06.960604 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:06.960581 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zjmvc" Jul 07 06:32:06.961913 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:06.961882 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacb18573_2d7c_4e7b_9429_dcad9be2ced6.slice/crio-3718fe3b5acbff4c8e48429da5e67fdda91032b8b94fd1c40e7cc90b97551d51 WatchSource:0}: Error finding container 3718fe3b5acbff4c8e48429da5e67fdda91032b8b94fd1c40e7cc90b97551d51: Status 404 returned error can't find the container with id 3718fe3b5acbff4c8e48429da5e67fdda91032b8b94fd1c40e7cc90b97551d51 Jul 07 06:32:06.970935 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:06.970913 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39868a40_e8ab_4965_b28a_83e1a6a46162.slice/crio-53e7a3139ad147d3541397e27f265846ecf3334874da73fed110746bc7a5bfa2 WatchSource:0}: Error finding container 53e7a3139ad147d3541397e27f265846ecf3334874da73fed110746bc7a5bfa2: Status 404 returned error can't find the container with id 53e7a3139ad147d3541397e27f265846ecf3334874da73fed110746bc7a5bfa2 Jul 07 06:32:07.209674 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:07.209591 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:07.209885 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:07.209756 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 06:32:07.209885 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:07.209844 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret podName:997fb304-b3ae-42e0-8a84-db42daea16b1 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:08.209819479 +0000 UTC m=+4.049660440 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret") pod "global-pull-secret-syncer-4bdk4" (UID: "997fb304-b3ae-42e0-8a84-db42daea16b1") : object "kube-system"/"original-pull-secret" not registered Jul 07 06:32:07.310776 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:07.310743 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs\") pod \"network-metrics-daemon-thndk\" (UID: \"e3255519-1e37-4781-8d6e-14394758a8b2\") " pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:07.310914 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:07.310804 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsp8h\" (UniqueName: \"kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h\") pod \"network-check-target-mqsn2\" (UID: \"d592a475-9102-419a-a6e0-5877d1838ca9\") " pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:07.310914 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:07.310904 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:32:07.311001 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:07.310916 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 06:32:07.311001 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:07.310934 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 06:32:07.311001 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:07.310946 2579 projected.go:196] Error preparing data for projected volume kube-api-access-vsp8h for pod openshift-network-diagnostics/network-check-target-mqsn2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:32:07.311001 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:07.310978 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs podName:e3255519-1e37-4781-8d6e-14394758a8b2 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:08.310957939 +0000 UTC m=+4.150798890 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs") pod "network-metrics-daemon-thndk" (UID: "e3255519-1e37-4781-8d6e-14394758a8b2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:32:07.311001 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:07.310995 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h podName:d592a475-9102-419a-a6e0-5877d1838ca9 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:08.310988264 +0000 UTC m=+4.150829211 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsp8h" (UniqueName: "kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h") pod "network-check-target-mqsn2" (UID: "d592a475-9102-419a-a6e0-5877d1838ca9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:32:07.632983 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:07.632809 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 06:27:07 +0000 UTC" deadline="2028-02-03 05:27:34.52623593 +0000 UTC" Jul 07 06:32:07.632983 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:07.632877 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13822h55m26.89336349s" Jul 07 06:32:07.777744 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:07.777712 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:07.777920 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:07.777835 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:07.796844 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:07.796780 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zjmvc" event={"ID":"39868a40-e8ab-4965-b28a-83e1a6a46162","Type":"ContainerStarted","Data":"53e7a3139ad147d3541397e27f265846ecf3334874da73fed110746bc7a5bfa2"} Jul 07 06:32:07.800470 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:07.800392 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" event={"ID":"b25f9d15-2923-4c0a-a8dc-c7941ce430a0","Type":"ContainerStarted","Data":"931065cdaebec02d2c931f4c3c03347c2513e3eb2d02f0888f964524ae55d058"} Jul 07 06:32:07.801942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:07.801912 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-86q7n" event={"ID":"d8deda72-2465-458f-af82-23eb7bd4156f","Type":"ContainerStarted","Data":"f3bae7c369235d1e17ea074a272a8c30a7b63a14ee72ac1c615aed7b14ad9a3a"} Jul 07 06:32:07.804829 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:07.804798 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9mqv" event={"ID":"acb18573-2d7c-4e7b-9429-dcad9be2ced6","Type":"ContainerStarted","Data":"3718fe3b5acbff4c8e48429da5e67fdda91032b8b94fd1c40e7cc90b97551d51"} Jul 07 06:32:07.809569 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:07.809502 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vkw7h" event={"ID":"ca16c608-dfd6-48a9-84c6-6a35f862c77a","Type":"ContainerStarted","Data":"10005e8cbc8b740386532359d0d89614d6d8d1902de2bd86494128d3e317713f"} Jul 07 06:32:07.812824 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:07.812758 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j6ngv" event={"ID":"228e7186-241d-4d93-9874-ee588f0a5e1d","Type":"ContainerStarted","Data":"9728808c2b40030af233d0ef8f0c9ca298c060a8c6c670a76449a420ad8719f3"} Jul 07 06:32:07.816672 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:07.816554 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" event={"ID":"a328cd98-578d-4f2a-a622-0739620a9a6f","Type":"ContainerStarted","Data":"a0b20da210b91b7b09cf77393483421a5e4942c5c50fecc5e3511423f2b1faed"} Jul 07 06:32:07.819660 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:07.819590 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5q8w9" event={"ID":"606bdf3e-ea47-4a60-a2fd-cf5aa63d341c","Type":"ContainerStarted","Data":"80c918b53e332b7a55c67fcd9a0dd3a77265cb14ab56d9e6093b1c5966b4d5f9"} Jul 07 06:32:07.823068 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:07.823013 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8kknw" event={"ID":"e778e873-cfa7-4f30-9190-d2cfa231a778","Type":"ContainerStarted","Data":"cc99ba9dde7241a2aee8ec8fa480df805768aa019210710afe94c4bfc47f0f70"} Jul 07 06:32:08.218585 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:08.217900 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:08.218585 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:08.218120 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 06:32:08.218585 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:08.218189 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret podName:997fb304-b3ae-42e0-8a84-db42daea16b1 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:10.218169365 +0000 UTC m=+6.058010319 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret") pod "global-pull-secret-syncer-4bdk4" (UID: "997fb304-b3ae-42e0-8a84-db42daea16b1") : object "kube-system"/"original-pull-secret" not registered Jul 07 06:32:08.320172 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:08.319303 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs\") pod \"network-metrics-daemon-thndk\" (UID: \"e3255519-1e37-4781-8d6e-14394758a8b2\") " pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:08.320172 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:08.319379 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsp8h\" (UniqueName: \"kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h\") pod \"network-check-target-mqsn2\" (UID: \"d592a475-9102-419a-a6e0-5877d1838ca9\") " pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:08.320172 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:08.319554 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 06:32:08.320172 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:08.319577 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 06:32:08.320172 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:08.319592 2579 projected.go:196] Error preparing data for projected volume kube-api-access-vsp8h for pod openshift-network-diagnostics/network-check-target-mqsn2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:32:08.320172 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:08.319651 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h podName:d592a475-9102-419a-a6e0-5877d1838ca9 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:10.31963232 +0000 UTC m=+6.159473292 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsp8h" (UniqueName: "kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h") pod "network-check-target-mqsn2" (UID: "d592a475-9102-419a-a6e0-5877d1838ca9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:32:08.320172 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:08.320083 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:32:08.320172 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:08.320136 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs podName:e3255519-1e37-4781-8d6e-14394758a8b2 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:10.320119669 +0000 UTC m=+6.159960632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs") pod "network-metrics-daemon-thndk" (UID: "e3255519-1e37-4781-8d6e-14394758a8b2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:32:08.633488 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:08.633380 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 06:27:07 +0000 UTC" deadline="2028-01-14 20:21:39.772980085 +0000 UTC" Jul 07 06:32:08.633488 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:08.633424 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13357h49m31.139561935s" Jul 07 06:32:08.784915 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:08.783280 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:08.784915 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:08.783427 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:08.784915 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:08.783898 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:08.784915 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:08.784000 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:09.777273 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:09.777231 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:09.777812 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:09.777367 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:10.235633 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:10.235542 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:10.235822 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:10.235690 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 06:32:10.235822 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:10.235758 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret podName:997fb304-b3ae-42e0-8a84-db42daea16b1 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:14.235737882 +0000 UTC m=+10.075578828 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret") pod "global-pull-secret-syncer-4bdk4" (UID: "997fb304-b3ae-42e0-8a84-db42daea16b1") : object "kube-system"/"original-pull-secret" not registered Jul 07 06:32:10.337120 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:10.336918 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs\") pod \"network-metrics-daemon-thndk\" (UID: \"e3255519-1e37-4781-8d6e-14394758a8b2\") " pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:10.337120 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:10.337019 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsp8h\" (UniqueName: \"kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h\") pod \"network-check-target-mqsn2\" (UID: \"d592a475-9102-419a-a6e0-5877d1838ca9\") " pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:10.337120 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:10.337109 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:32:10.337446 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:10.337205 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs podName:e3255519-1e37-4781-8d6e-14394758a8b2 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:14.337180865 +0000 UTC m=+10.177021812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs") pod "network-metrics-daemon-thndk" (UID: "e3255519-1e37-4781-8d6e-14394758a8b2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:32:10.337446 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:10.337221 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 06:32:10.337446 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:10.337247 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 06:32:10.337446 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:10.337261 2579 projected.go:196] Error preparing data for projected volume kube-api-access-vsp8h for pod openshift-network-diagnostics/network-check-target-mqsn2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:32:10.337446 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:10.337314 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h podName:d592a475-9102-419a-a6e0-5877d1838ca9 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:14.337297773 +0000 UTC m=+10.177138733 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsp8h" (UniqueName: "kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h") pod "network-check-target-mqsn2" (UID: "d592a475-9102-419a-a6e0-5877d1838ca9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:32:10.778938 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:10.778409 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:10.778938 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:10.778571 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:10.778938 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:10.778613 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:10.778938 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:10.778712 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:11.777885 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:11.777596 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:11.777885 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:11.777739 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:12.778116 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:12.778029 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:12.778116 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:12.778088 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:12.778691 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:12.778187 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:12.778691 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:12.778340 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:13.777777 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:13.777588 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:13.777777 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:13.777730 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:14.271994 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:14.271935 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:14.272505 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:14.272162 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 06:32:14.272505 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:14.272403 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret podName:997fb304-b3ae-42e0-8a84-db42daea16b1 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:22.272366473 +0000 UTC m=+18.112207438 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret") pod "global-pull-secret-syncer-4bdk4" (UID: "997fb304-b3ae-42e0-8a84-db42daea16b1") : object "kube-system"/"original-pull-secret" not registered Jul 07 06:32:14.372930 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:14.372877 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsp8h\" (UniqueName: \"kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h\") pod \"network-check-target-mqsn2\" (UID: \"d592a475-9102-419a-a6e0-5877d1838ca9\") " pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:14.373123 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:14.372972 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs\") pod \"network-metrics-daemon-thndk\" (UID: \"e3255519-1e37-4781-8d6e-14394758a8b2\") " pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:14.373123 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:14.373100 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:32:14.373231 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:14.373118 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 06:32:14.373231 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:14.373150 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 06:32:14.373231 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:14.373163 2579 projected.go:196] Error preparing data for projected volume kube-api-access-vsp8h for pod openshift-network-diagnostics/network-check-target-mqsn2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:32:14.373231 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:14.373170 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs podName:e3255519-1e37-4781-8d6e-14394758a8b2 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:22.373146378 +0000 UTC m=+18.212987344 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs") pod "network-metrics-daemon-thndk" (UID: "e3255519-1e37-4781-8d6e-14394758a8b2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:32:14.373231 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:14.373220 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h podName:d592a475-9102-419a-a6e0-5877d1838ca9 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:22.373201948 +0000 UTC m=+18.213042909 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsp8h" (UniqueName: "kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h") pod "network-check-target-mqsn2" (UID: "d592a475-9102-419a-a6e0-5877d1838ca9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:32:14.778082 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:14.778050 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:14.778261 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:14.778181 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:14.778330 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:14.778259 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:14.778330 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:14.778256 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:15.777215 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:15.777125 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:15.777663 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:15.777266 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:16.781263 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:16.780565 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:16.781263 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:16.780688 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:16.781263 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:16.781047 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:16.781263 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:16.781147 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:17.777872 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:17.777835 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:17.778049 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:17.777970 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:18.779986 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:18.779953 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:18.779986 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:18.779972 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:18.780445 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:18.780066 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:18.780445 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:18.780159 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:19.777723 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:19.777687 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:19.777895 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:19.777817 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:20.777698 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:20.777665 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:20.777698 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:20.777692 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:20.778158 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:20.777811 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:20.778158 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:20.777921 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:21.777027 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:21.776993 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:21.777208 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:21.777117 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:22.333944 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:22.333901 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:22.334407 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:22.334033 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 06:32:22.334407 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:22.334116 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret podName:997fb304-b3ae-42e0-8a84-db42daea16b1 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:38.334090814 +0000 UTC m=+34.173931762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret") pod "global-pull-secret-syncer-4bdk4" (UID: "997fb304-b3ae-42e0-8a84-db42daea16b1") : object "kube-system"/"original-pull-secret" not registered Jul 07 06:32:22.434675 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:22.434449 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs\") pod \"network-metrics-daemon-thndk\" (UID: \"e3255519-1e37-4781-8d6e-14394758a8b2\") " pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:22.434859 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:22.434727 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsp8h\" (UniqueName: \"kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h\") pod \"network-check-target-mqsn2\" (UID: \"d592a475-9102-419a-a6e0-5877d1838ca9\") " pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:22.434859 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:22.434600 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:32:22.434859 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:22.434817 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs podName:e3255519-1e37-4781-8d6e-14394758a8b2 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:38.434795113 +0000 UTC m=+34.274636076 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs") pod "network-metrics-daemon-thndk" (UID: "e3255519-1e37-4781-8d6e-14394758a8b2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:32:22.435036 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:22.434878 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 06:32:22.435036 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:22.434899 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 06:32:22.435036 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:22.434913 2579 projected.go:196] Error preparing data for projected volume kube-api-access-vsp8h for pod openshift-network-diagnostics/network-check-target-mqsn2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:32:22.435036 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:22.435004 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h podName:d592a475-9102-419a-a6e0-5877d1838ca9 nodeName:}" failed. No retries permitted until 2026-07-07 06:32:38.434988886 +0000 UTC m=+34.274829853 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsp8h" (UniqueName: "kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h") pod "network-check-target-mqsn2" (UID: "d592a475-9102-419a-a6e0-5877d1838ca9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:32:22.776948 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:22.776917 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:22.777109 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:22.776927 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:22.777109 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:22.777059 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:22.777196 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:22.777127 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:23.777005 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:23.776967 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:23.777494 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:23.777093 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:24.777802 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:24.777765 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:24.778258 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:24.777884 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:24.778258 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:24.777941 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:24.778258 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:24.778044 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:25.777328 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:25.777295 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:25.777516 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:25.777418 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:26.778020 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:26.777842 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:26.778668 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:26.777876 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:26.778668 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:26.778104 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:26.778668 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:26.778145 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:26.862475 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:26.861791 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-176.ec2.internal" event={"ID":"b737b79e957a55e359ff156469d79a5e","Type":"ContainerStarted","Data":"aeb1e7d787497e33eb6640bfde076c5a877166580eda5d8a3411be1765375d59"} Jul 07 06:32:26.864515 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:26.864483 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j6ngv" event={"ID":"228e7186-241d-4d93-9874-ee588f0a5e1d","Type":"ContainerStarted","Data":"654d016c1f148c8461dd16b701dcaf1fa43317181ee37312b4b2eaa9805d3d13"} Jul 07 06:32:26.866225 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:26.866209 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 06:32:26.866579 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:26.866561 2579 generic.go:356] "Generic (PLEG): container finished" podID="a328cd98-578d-4f2a-a622-0739620a9a6f" containerID="8031f2c2b12bce89b29f3a9a84bbf7d8fc57d0cab9804f9ef15680ec18611bd2" exitCode=1 Jul 07 06:32:26.866665 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:26.866618 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" event={"ID":"a328cd98-578d-4f2a-a622-0739620a9a6f","Type":"ContainerStarted","Data":"1f0d5851ed2290cd6f0aa827ba2a7857fd3c711cc26b8d8ec71c8c9debe706f7"} Jul 07 06:32:26.866758 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:26.866701 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" event={"ID":"a328cd98-578d-4f2a-a622-0739620a9a6f","Type":"ContainerDied","Data":"8031f2c2b12bce89b29f3a9a84bbf7d8fc57d0cab9804f9ef15680ec18611bd2"} Jul 07 06:32:26.866814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:26.866758 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" event={"ID":"a328cd98-578d-4f2a-a622-0739620a9a6f","Type":"ContainerStarted","Data":"4b573ece5b63471ba5524da34e98586ef3e87cc62a63425ac23cef2ae1e6cec1"} Jul 07 06:32:26.867888 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:26.867863 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8kknw" event={"ID":"e778e873-cfa7-4f30-9190-d2cfa231a778","Type":"ContainerStarted","Data":"37a1398054e421239494ce0871d4945ac71942fa577a7a7206bac7bf786f9780"} Jul 07 06:32:26.876944 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:26.876902 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-176.ec2.internal" podStartSLOduration=20.87688584 podStartE2EDuration="20.87688584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:32:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:32:26.87614848 +0000 UTC m=+22.715989448" watchObservedRunningTime="2026-07-07 06:32:26.87688584 +0000 UTC m=+22.716726808" Jul 07 06:32:26.893287 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:26.893249 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-j6ngv" podStartSLOduration=2.313894609 podStartE2EDuration="21.893239946s" totalImagesPullingTime="19.579345337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:32:26.892812039 +0000 UTC m=+22.732653006" watchObservedRunningTime="2026-07-07 06:32:26.893239946 +0000 UTC m=+22.733080912" Jul 07 06:32:26.910184 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:26.910124 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8kknw" podStartSLOduration=2.46305537 podStartE2EDuration="21.910112076s" totalImagesPullingTime="19.447056706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:32:26.909718116 +0000 UTC m=+22.749559083" watchObservedRunningTime="2026-07-07 06:32:26.910112076 +0000 UTC m=+22.749953043" Jul 07 06:32:27.777195 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.777159 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:27.777411 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:27.777257 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:27.871436 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.871344 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zjmvc" event={"ID":"39868a40-e8ab-4965-b28a-83e1a6a46162","Type":"ContainerStarted","Data":"4858eb28817e584defd4d04953e47943cc371c78ffa0c9fcaa769c8f901f5f6a"} Jul 07 06:32:27.872993 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.872968 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" event={"ID":"b25f9d15-2923-4c0a-a8dc-c7941ce430a0","Type":"ContainerStarted","Data":"52c6b37122c66e7f87150260678221854e25913e5d56a4bdbd4fef20dad0e9fa"} Jul 07 06:32:27.874350 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.874324 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-86q7n" event={"ID":"d8deda72-2465-458f-af82-23eb7bd4156f","Type":"ContainerStarted","Data":"617caaa4346045426579ec50d33a5aaf86dc6170b55c128bf0a62e49d0c13d09"} Jul 07 06:32:27.875798 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.875773 2579 generic.go:356] "Generic (PLEG): container finished" podID="acb18573-2d7c-4e7b-9429-dcad9be2ced6" containerID="aa2faf95abd1ef8162fbc622824aae45362cfaae6ad37fd7686bd832c3ada910" exitCode=0 Jul 07 06:32:27.875899 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.875852 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9mqv" event={"ID":"acb18573-2d7c-4e7b-9429-dcad9be2ced6","Type":"ContainerDied","Data":"aa2faf95abd1ef8162fbc622824aae45362cfaae6ad37fd7686bd832c3ada910"} Jul 07 06:32:27.877251 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.877210 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vkw7h" event={"ID":"ca16c608-dfd6-48a9-84c6-6a35f862c77a","Type":"ContainerStarted","Data":"3df13508db391c9f2725e4eed50da5f621efdf2c18d7dde0b434bd8e88bd7b1c"} Jul 07 06:32:27.881340 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.881325 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 06:32:27.881701 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.881683 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" event={"ID":"a328cd98-578d-4f2a-a622-0739620a9a6f","Type":"ContainerStarted","Data":"2c7418888551c44c603d4a813536a5a081de07ad597ba6b7f3168153bbf67024"} Jul 07 06:32:27.881811 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.881753 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" event={"ID":"a328cd98-578d-4f2a-a622-0739620a9a6f","Type":"ContainerStarted","Data":"ef1a3f5426146a2f4be5f918938a882ebc889a414e5fce00be4834eebcf1e4e8"} Jul 07 06:32:27.881811 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.881806 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" event={"ID":"a328cd98-578d-4f2a-a622-0739620a9a6f","Type":"ContainerStarted","Data":"0af3d910f112d8c1fe01ae0fd870f56e2457231745ef3d758c6f1317ffdff07d"} Jul 07 06:32:27.883076 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.883054 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5q8w9" event={"ID":"606bdf3e-ea47-4a60-a2fd-cf5aa63d341c","Type":"ContainerStarted","Data":"2d1797dcf1493193ffd8312c28c412ab08021147139da8347ca186d47f6e8b35"} Jul 07 06:32:27.884404 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.884384 2579 generic.go:356] "Generic (PLEG): container finished" podID="f47328e32847cb792862c0d80bc10faf" containerID="cad386c93fc17bb4b63bb736be2d9e5687800627a9f5f5f866713c5341a4c529" exitCode=0 Jul 07 06:32:27.884492 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.884444 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal" event={"ID":"f47328e32847cb792862c0d80bc10faf","Type":"ContainerDied","Data":"cad386c93fc17bb4b63bb736be2d9e5687800627a9f5f5f866713c5341a4c529"} Jul 07 06:32:27.903684 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.903643 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zjmvc" podStartSLOduration=3.558429405 podStartE2EDuration="22.903629068s" totalImagesPullingTime="19.345199663s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:32:27.887580049 +0000 UTC m=+23.727421016" watchObservedRunningTime="2026-07-07 06:32:27.903629068 +0000 UTC m=+23.743470034" Jul 07 06:32:27.920080 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.920034 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-86q7n" podStartSLOduration=3.897724855 podStartE2EDuration="22.920016509s" totalImagesPullingTime="19.022291654s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:32:27.91982284 +0000 UTC m=+23.759663807" watchObservedRunningTime="2026-07-07 06:32:27.920016509 +0000 UTC m=+23.759857477" Jul 07 06:32:27.935119 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.935084 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5q8w9" podStartSLOduration=11.844737182 podStartE2EDuration="22.935070732s" totalImagesPullingTime="11.09033355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:32:27.934524404 +0000 UTC m=+23.774365372" watchObservedRunningTime="2026-07-07 06:32:27.935070732 +0000 UTC m=+23.774911698" Jul 07 06:32:27.953444 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:27.953402 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vkw7h" podStartSLOduration=3.967475316 podStartE2EDuration="22.953385641s" totalImagesPullingTime="18.985910325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:32:27.952853642 +0000 UTC m=+23.792694610" watchObservedRunningTime="2026-07-07 06:32:27.953385641 +0000 UTC m=+23.793226608" Jul 07 06:32:28.075531 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:28.075492 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:28.415603 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:28.415577 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 06:32:28.680244 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:28.680138 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:28.683810 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:28.683729 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T06:32:28.415598877Z","UUID":"6891016f-26a8-4199-95ab-32cb8ac0c006","Handler":null,"Name":"","Endpoint":""} Jul 07 06:32:28.685860 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:28.685836 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 06:32:28.685964 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:28.685877 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 06:32:28.777325 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:28.777289 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:28.777563 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:28.777289 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:28.777563 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:28.777456 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:28.777563 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:28.777492 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:28.889308 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:28.889233 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal" event={"ID":"f47328e32847cb792862c0d80bc10faf","Type":"ContainerStarted","Data":"5c64bb9cdc22c071b4b4a0e7e6cf83778954e8cfcb72cb9e8e24df93362785b5"} Jul 07 06:32:28.891358 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:28.891328 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" event={"ID":"b25f9d15-2923-4c0a-a8dc-c7941ce430a0","Type":"ContainerStarted","Data":"7a819af5fb864363c0a8812111aa647fb1b3c4348cd0d50dc1266ddc7aecb165"} Jul 07 06:32:28.905425 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:28.904657 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-176.ec2.internal" podStartSLOduration=22.904638824 podStartE2EDuration="22.904638824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:32:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:32:28.904492652 +0000 UTC m=+24.744333620" watchObservedRunningTime="2026-07-07 06:32:28.904638824 +0000 UTC m=+24.744479793" Jul 07 06:32:29.275821 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:29.275792 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:29.483556 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:29.483464 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:29.777492 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:29.777464 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:29.777707 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:29.777571 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:29.878762 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:29.878735 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:29.897402 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:29.897357 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 06:32:29.897837 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:29.897814 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" event={"ID":"a328cd98-578d-4f2a-a622-0739620a9a6f","Type":"ContainerStarted","Data":"8f40e0fd0d865f3a5910302cc6d306ef2afedad6aa1403cd284afcf85fb1fcf9"} Jul 07 06:32:29.899891 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:29.899855 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" event={"ID":"b25f9d15-2923-4c0a-a8dc-c7941ce430a0","Type":"ContainerStarted","Data":"5afcedd02f9bbe57978116ef6224931b94a3a57a69233586e3b3b06329831a7c"} Jul 07 06:32:29.919467 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:29.919422 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-srfjx" podStartSLOduration=3.051075471 podStartE2EDuration="24.919407958s" totalImagesPullingTime="21.868332487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:32:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:32:29.919259717 +0000 UTC m=+25.759100710" watchObservedRunningTime="2026-07-07 06:32:29.919407958 +0000 UTC m=+25.759248924" Jul 07 06:32:30.077597 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:30.077515 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:30.262899 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:30.262865 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:30.674982 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:30.674951 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:30.777759 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:30.777723 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:30.777931 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:30.777723 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:30.777931 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:30.777870 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:30.777931 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:30.777895 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:30.814940 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:30.814909 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5q8w9" Jul 07 06:32:30.815436 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:30.815419 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5q8w9" Jul 07 06:32:30.890636 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:30.890603 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:30.902224 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:30.902199 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5q8w9" Jul 07 06:32:30.902631 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:30.902616 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5q8w9" Jul 07 06:32:31.082634 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:31.082602 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:31.481970 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:31.481897 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:31.777352 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:31.777323 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:31.777510 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:31.777469 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:31.818398 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:31.818354 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:32.777534 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:32.777511 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:32.777936 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:32.777643 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:32.777936 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:32.777704 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:32.777936 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:32.777791 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:32.910612 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:32.910308 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 06:32:32.911217 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:32.911040 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" event={"ID":"a328cd98-578d-4f2a-a622-0739620a9a6f","Type":"ContainerStarted","Data":"6939583538f64303423a273668c6e4eef9f8c7849b90d674395ff939c53b7592"} Jul 07 06:32:32.911648 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:32.911546 2579 scope.go:117] "RemoveContainer" containerID="8031f2c2b12bce89b29f3a9a84bbf7d8fc57d0cab9804f9ef15680ec18611bd2" Jul 07 06:32:33.777086 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:33.777067 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:33.777193 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:33.777159 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:33.914460 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:33.914421 2579 generic.go:356] "Generic (PLEG): container finished" podID="acb18573-2d7c-4e7b-9429-dcad9be2ced6" containerID="305e72d053d95cce4a9f8191190f7bddd11dfa6e26c0d890bb10628ca523951c" exitCode=0 Jul 07 06:32:33.915241 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:33.914507 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9mqv" event={"ID":"acb18573-2d7c-4e7b-9429-dcad9be2ced6","Type":"ContainerDied","Data":"305e72d053d95cce4a9f8191190f7bddd11dfa6e26c0d890bb10628ca523951c"} Jul 07 06:32:33.918347 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:33.918331 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 06:32:33.918701 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:33.918682 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" event={"ID":"a328cd98-578d-4f2a-a622-0739620a9a6f","Type":"ContainerStarted","Data":"93fb69bc5653a6a86138e4a4ea0fa967d1ad5a0d44e6f46738bb11d3a0f9c698"} Jul 07 06:32:33.918918 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:33.918904 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:33.918981 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:33.918959 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:33.919033 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:33.919006 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:33.935114 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:33.935088 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:33.935223 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:33.935196 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:32:33.970520 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:33.970468 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" podStartSLOduration=9.531739568999999 podStartE2EDuration="28.970454878s" totalImagesPullingTime="19.438715309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:32:33.968970249 +0000 UTC m=+29.808811215" watchObservedRunningTime="2026-07-07 06:32:33.970454878 +0000 UTC m=+29.810295849" Jul 07 06:32:34.778461 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:34.778428 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:34.778607 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:34.778539 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:34.778607 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:34.778589 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:34.778699 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:34.778680 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:35.777642 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:35.777465 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:35.778015 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:35.777715 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:35.924396 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:35.924352 2579 generic.go:356] "Generic (PLEG): container finished" podID="acb18573-2d7c-4e7b-9429-dcad9be2ced6" containerID="6bde64b59fdd570de3e430e58a09a90d02f19368e2a31c5ac2eb0e95f6de41a7" exitCode=0 Jul 07 06:32:35.924523 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:35.924446 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9mqv" event={"ID":"acb18573-2d7c-4e7b-9429-dcad9be2ced6","Type":"ContainerDied","Data":"6bde64b59fdd570de3e430e58a09a90d02f19368e2a31c5ac2eb0e95f6de41a7"} Jul 07 06:32:35.988448 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:35.988406 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:36.108008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:36.107915 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:36.283991 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:36.283958 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:36.496583 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:36.496509 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:36.574861 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:36.574799 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:36.777757 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:36.777727 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:36.778106 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:36.777835 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:36.778106 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:36.777900 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:36.778106 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:36.778011 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:36.869257 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:36.869202 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:36.932752 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:36.932718 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:37.073921 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:37.073851 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:37.174175 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:37.174144 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:37.488992 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:37.488950 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:37.590861 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:37.590782 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:37.691907 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:37.691874 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:37.777584 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:37.777558 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:37.777734 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:37.777646 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:37.880786 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:37.880719 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:37.931590 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:37.931560 2579 generic.go:356] "Generic (PLEG): container finished" podID="acb18573-2d7c-4e7b-9429-dcad9be2ced6" containerID="e2ac21d26a085d7b282062accd02e8f5f35fb17a3fe0213f42f9fda10afe8526" exitCode=0 Jul 07 06:32:37.931709 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:37.931620 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9mqv" event={"ID":"acb18573-2d7c-4e7b-9429-dcad9be2ced6","Type":"ContainerDied","Data":"e2ac21d26a085d7b282062accd02e8f5f35fb17a3fe0213f42f9fda10afe8526"} Jul 07 06:32:38.144767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:38.144687 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:38.348168 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:38.348136 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:38.348338 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:38.348266 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 06:32:38.348338 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:38.348320 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret podName:997fb304-b3ae-42e0-8a84-db42daea16b1 nodeName:}" failed. No retries permitted until 2026-07-07 06:33:10.348306962 +0000 UTC m=+66.188147921 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret") pod "global-pull-secret-syncer-4bdk4" (UID: "997fb304-b3ae-42e0-8a84-db42daea16b1") : object "kube-system"/"original-pull-secret" not registered Jul 07 06:32:38.449202 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:38.449128 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs\") pod \"network-metrics-daemon-thndk\" (UID: \"e3255519-1e37-4781-8d6e-14394758a8b2\") " pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:38.449202 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:38.449170 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsp8h\" (UniqueName: \"kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h\") pod \"network-check-target-mqsn2\" (UID: \"d592a475-9102-419a-a6e0-5877d1838ca9\") " pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:38.449384 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:38.449258 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:32:38.449384 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:38.449266 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 06:32:38.449384 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:38.449282 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 06:32:38.449384 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:38.449290 2579 projected.go:196] Error preparing data for projected volume kube-api-access-vsp8h for pod openshift-network-diagnostics/network-check-target-mqsn2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:32:38.449384 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:38.449308 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs podName:e3255519-1e37-4781-8d6e-14394758a8b2 nodeName:}" failed. No retries permitted until 2026-07-07 06:33:10.44929456 +0000 UTC m=+66.289135504 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs") pod "network-metrics-daemon-thndk" (UID: "e3255519-1e37-4781-8d6e-14394758a8b2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 06:32:38.449384 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:38.449325 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h podName:d592a475-9102-419a-a6e0-5877d1838ca9 nodeName:}" failed. No retries permitted until 2026-07-07 06:33:10.449314052 +0000 UTC m=+66.289154997 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsp8h" (UniqueName: "kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h") pod "network-check-target-mqsn2" (UID: "d592a475-9102-419a-a6e0-5877d1838ca9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 06:32:38.777322 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:38.777295 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:38.777509 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:38.777438 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:38.777586 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:38.777510 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:38.777636 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:38.777610 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:39.088258 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:39.088175 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:39.188687 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:39.188656 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:39.777323 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:39.777288 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:39.777595 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:39.777412 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:39.894201 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:39.894170 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:40.031978 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:40.031900 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:40.135390 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:40.135334 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:40.233518 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:40.233476 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:40.332328 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:40.332256 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:40.777063 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:40.777037 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:40.777185 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:40.777145 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:40.777234 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:40.777201 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:40.777344 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:40.777322 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:40.875582 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:40.875555 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:40.957776 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:40.957741 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4bdk4"] Jul 07 06:32:40.957941 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:40.957845 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:40.957941 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:40.957934 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:40.962040 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:40.962016 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-thndk"] Jul 07 06:32:40.962158 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:40.962112 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:40.962238 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:40.962218 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:40.962718 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:40.962695 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mqsn2"] Jul 07 06:32:40.962830 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:40.962775 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:40.962891 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:40.962859 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:41.219673 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:41.219636 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:41.320344 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:41.320311 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:42.777679 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:42.777651 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:42.777679 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:42.777669 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:42.778179 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:42.777669 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:42.778179 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:42.777779 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:42.778179 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:42.777859 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:42.778179 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:42.777933 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:43.879356 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:43.879184 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:43.947887 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:43.947857 2579 generic.go:356] "Generic (PLEG): container finished" podID="acb18573-2d7c-4e7b-9429-dcad9be2ced6" containerID="28a42ae65846d08bf9c3c1c2e91b6edcb07f98c5a92ea67aad71e527bd7481d3" exitCode=0 Jul 07 06:32:43.948041 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:43.947896 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9mqv" event={"ID":"acb18573-2d7c-4e7b-9429-dcad9be2ced6","Type":"ContainerDied","Data":"28a42ae65846d08bf9c3c1c2e91b6edcb07f98c5a92ea67aad71e527bd7481d3"} Jul 07 06:32:43.978645 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:43.978612 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:44.778875 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:44.778837 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:44.779037 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:44.778930 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:44.779037 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:44.778949 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4bdk4" podUID="997fb304-b3ae-42e0-8a84-db42daea16b1" Jul 07 06:32:44.779037 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:44.778983 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:44.779138 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:44.779048 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mqsn2" podUID="d592a475-9102-419a-a6e0-5877d1838ca9" Jul 07 06:32:44.779138 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:32:44.779121 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-thndk" podUID="e3255519-1e37-4781-8d6e-14394758a8b2" Jul 07 06:32:44.952713 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:44.952684 2579 generic.go:356] "Generic (PLEG): container finished" podID="acb18573-2d7c-4e7b-9429-dcad9be2ced6" containerID="a5122744a95d9633c24be5986d2fa0ed63763b8c098796ab7fb2653ec734db22" exitCode=0 Jul 07 06:32:44.953066 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:44.952722 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9mqv" event={"ID":"acb18573-2d7c-4e7b-9429-dcad9be2ced6","Type":"ContainerDied","Data":"a5122744a95d9633c24be5986d2fa0ed63763b8c098796ab7fb2653ec734db22"} Jul 07 06:32:45.428288 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.428216 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-176.ec2.internal" event="NodeReady" Jul 07 06:32:45.428443 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.428329 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 06:32:45.462123 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.462095 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5658684c68-khz7t"] Jul 07 06:32:45.476180 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.476153 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sgzm5"] Jul 07 06:32:45.476284 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.476259 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.479262 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.479235 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 06:32:45.479358 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.479307 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 06:32:45.479358 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.479325 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-htvtq\"" Jul 07 06:32:45.479560 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.479545 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 06:32:45.485174 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.485141 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 06:32:45.492913 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.492892 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5658684c68-khz7t"] Jul 07 06:32:45.493002 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.492940 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hnq4d"] Jul 07 06:32:45.493057 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.493027 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sgzm5" Jul 07 06:32:45.495844 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.495826 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 06:32:45.495844 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.495840 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lfbmp\"" Jul 07 06:32:45.495984 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.495840 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 06:32:45.513064 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.513040 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sgzm5"] Jul 07 06:32:45.513146 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.513070 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hnq4d"] Jul 07 06:32:45.513191 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.513147 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.515824 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.515804 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 06:32:45.515923 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.515900 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 06:32:45.516244 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.516227 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 06:32:45.516574 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.516557 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 06:32:45.516656 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.516643 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-68w6b\"" Jul 07 06:32:45.576124 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.576096 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vlbw7"] Jul 07 06:32:45.603194 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603162 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5c4510a-0d58-4dcb-924d-47a74eaaa9b3-data-volume\") pod \"insights-runtime-extractor-hnq4d\" (UID: \"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3\") " pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.603194 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603192 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfqjx\" (UniqueName: \"kubernetes.io/projected/9bf0c340-1448-46eb-aa8a-a8c1c9738ba7-kube-api-access-xfqjx\") pod \"dns-default-sgzm5\" (UID: \"9bf0c340-1448-46eb-aa8a-a8c1c9738ba7\") " pod="openshift-dns/dns-default-sgzm5" Jul 07 06:32:45.603411 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603257 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab14650a-fd9f-4605-ac6c-7435637be8e7-ca-trust-extracted\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.603411 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603285 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab14650a-fd9f-4605-ac6c-7435637be8e7-bound-sa-token\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.603411 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603306 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5c4510a-0d58-4dcb-924d-47a74eaaa9b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnq4d\" (UID: \"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3\") " pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.603411 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603337 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab14650a-fd9f-4605-ac6c-7435637be8e7-image-registry-private-configuration\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.603411 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603352 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab14650a-fd9f-4605-ac6c-7435637be8e7-registry-certificates\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.603411 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603387 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab14650a-fd9f-4605-ac6c-7435637be8e7-trusted-ca\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.603608 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603426 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab14650a-fd9f-4605-ac6c-7435637be8e7-installation-pull-secrets\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.603608 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603469 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5c4510a-0d58-4dcb-924d-47a74eaaa9b3-crio-socket\") pod \"insights-runtime-extractor-hnq4d\" (UID: \"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3\") " pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.603608 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603488 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf0c340-1448-46eb-aa8a-a8c1c9738ba7-config-volume\") pod \"dns-default-sgzm5\" (UID: \"9bf0c340-1448-46eb-aa8a-a8c1c9738ba7\") " pod="openshift-dns/dns-default-sgzm5" Jul 07 06:32:45.603608 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603503 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab14650a-fd9f-4605-ac6c-7435637be8e7-registry-tls\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.603608 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603517 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npbh5\" (UniqueName: \"kubernetes.io/projected/ab14650a-fd9f-4605-ac6c-7435637be8e7-kube-api-access-npbh5\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.603608 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603539 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5c4510a-0d58-4dcb-924d-47a74eaaa9b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnq4d\" (UID: \"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3\") " pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.603608 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603561 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqbqs\" (UniqueName: \"kubernetes.io/projected/f5c4510a-0d58-4dcb-924d-47a74eaaa9b3-kube-api-access-kqbqs\") pod \"insights-runtime-extractor-hnq4d\" (UID: \"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3\") " pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.603608 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603579 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bf0c340-1448-46eb-aa8a-a8c1c9738ba7-metrics-tls\") pod \"dns-default-sgzm5\" (UID: \"9bf0c340-1448-46eb-aa8a-a8c1c9738ba7\") " pod="openshift-dns/dns-default-sgzm5" Jul 07 06:32:45.603608 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.603592 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bf0c340-1448-46eb-aa8a-a8c1c9738ba7-tmp-dir\") pod \"dns-default-sgzm5\" (UID: \"9bf0c340-1448-46eb-aa8a-a8c1c9738ba7\") " pod="openshift-dns/dns-default-sgzm5" Jul 07 06:32:45.611356 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.611332 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vlbw7"] Jul 07 06:32:45.611483 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.611426 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vlbw7" Jul 07 06:32:45.614139 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.614120 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 06:32:45.614238 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.614123 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bdntf\"" Jul 07 06:32:45.614296 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.614123 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 06:32:45.614397 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.614382 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 06:32:45.651639 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.651596 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:45.704444 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.704358 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab14650a-fd9f-4605-ac6c-7435637be8e7-installation-pull-secrets\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.704444 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.704407 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5c4510a-0d58-4dcb-924d-47a74eaaa9b3-crio-socket\") pod \"insights-runtime-extractor-hnq4d\" (UID: \"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3\") " pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.704614 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.704476 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf0c340-1448-46eb-aa8a-a8c1c9738ba7-config-volume\") pod \"dns-default-sgzm5\" (UID: \"9bf0c340-1448-46eb-aa8a-a8c1c9738ba7\") " pod="openshift-dns/dns-default-sgzm5" Jul 07 06:32:45.704614 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.704512 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab14650a-fd9f-4605-ac6c-7435637be8e7-registry-tls\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.704614 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.704536 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npbh5\" (UniqueName: \"kubernetes.io/projected/ab14650a-fd9f-4605-ac6c-7435637be8e7-kube-api-access-npbh5\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.704732 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.704656 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5c4510a-0d58-4dcb-924d-47a74eaaa9b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnq4d\" (UID: \"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3\") " pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.704732 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.704697 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqbqs\" (UniqueName: \"kubernetes.io/projected/f5c4510a-0d58-4dcb-924d-47a74eaaa9b3-kube-api-access-kqbqs\") pod \"insights-runtime-extractor-hnq4d\" (UID: \"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3\") " pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.704732 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.704723 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bf0c340-1448-46eb-aa8a-a8c1c9738ba7-metrics-tls\") pod \"dns-default-sgzm5\" (UID: \"9bf0c340-1448-46eb-aa8a-a8c1c9738ba7\") " pod="openshift-dns/dns-default-sgzm5" Jul 07 06:32:45.704870 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.704744 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bf0c340-1448-46eb-aa8a-a8c1c9738ba7-tmp-dir\") pod \"dns-default-sgzm5\" (UID: \"9bf0c340-1448-46eb-aa8a-a8c1c9738ba7\") " pod="openshift-dns/dns-default-sgzm5" Jul 07 06:32:45.704870 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.704784 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88f4bcf7-85d5-4270-86fd-576d5c2414c8-cert\") pod \"ingress-canary-vlbw7\" (UID: \"88f4bcf7-85d5-4270-86fd-576d5c2414c8\") " pod="openshift-ingress-canary/ingress-canary-vlbw7" Jul 07 06:32:45.704870 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.704669 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5c4510a-0d58-4dcb-924d-47a74eaaa9b3-crio-socket\") pod \"insights-runtime-extractor-hnq4d\" (UID: \"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3\") " pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.704870 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.704827 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8z4q\" (UniqueName: \"kubernetes.io/projected/88f4bcf7-85d5-4270-86fd-576d5c2414c8-kube-api-access-w8z4q\") pod \"ingress-canary-vlbw7\" (UID: \"88f4bcf7-85d5-4270-86fd-576d5c2414c8\") " pod="openshift-ingress-canary/ingress-canary-vlbw7" Jul 07 06:32:45.705069 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.704873 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5c4510a-0d58-4dcb-924d-47a74eaaa9b3-data-volume\") pod \"insights-runtime-extractor-hnq4d\" (UID: \"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3\") " pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.705069 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.704901 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfqjx\" (UniqueName: \"kubernetes.io/projected/9bf0c340-1448-46eb-aa8a-a8c1c9738ba7-kube-api-access-xfqjx\") pod \"dns-default-sgzm5\" (UID: \"9bf0c340-1448-46eb-aa8a-a8c1c9738ba7\") " pod="openshift-dns/dns-default-sgzm5" Jul 07 06:32:45.705169 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.705073 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf0c340-1448-46eb-aa8a-a8c1c9738ba7-config-volume\") pod \"dns-default-sgzm5\" (UID: \"9bf0c340-1448-46eb-aa8a-a8c1c9738ba7\") " pod="openshift-dns/dns-default-sgzm5" Jul 07 06:32:45.705169 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.705121 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bf0c340-1448-46eb-aa8a-a8c1c9738ba7-tmp-dir\") pod \"dns-default-sgzm5\" (UID: \"9bf0c340-1448-46eb-aa8a-a8c1c9738ba7\") " pod="openshift-dns/dns-default-sgzm5" Jul 07 06:32:45.705169 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.705139 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab14650a-fd9f-4605-ac6c-7435637be8e7-ca-trust-extracted\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.705169 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.705167 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab14650a-fd9f-4605-ac6c-7435637be8e7-bound-sa-token\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.705392 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.705189 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5c4510a-0d58-4dcb-924d-47a74eaaa9b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnq4d\" (UID: \"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3\") " pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.705392 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.705218 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab14650a-fd9f-4605-ac6c-7435637be8e7-image-registry-private-configuration\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.705392 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.705240 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab14650a-fd9f-4605-ac6c-7435637be8e7-registry-certificates\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.705392 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.705250 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5c4510a-0d58-4dcb-924d-47a74eaaa9b3-data-volume\") pod \"insights-runtime-extractor-hnq4d\" (UID: \"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3\") " pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.705392 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.705266 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab14650a-fd9f-4605-ac6c-7435637be8e7-trusted-ca\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.705392 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.705291 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5c4510a-0d58-4dcb-924d-47a74eaaa9b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnq4d\" (UID: \"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3\") " pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.705740 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.705572 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab14650a-fd9f-4605-ac6c-7435637be8e7-ca-trust-extracted\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.706088 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.706067 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab14650a-fd9f-4605-ac6c-7435637be8e7-registry-certificates\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.706263 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.706244 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab14650a-fd9f-4605-ac6c-7435637be8e7-trusted-ca\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.708978 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.708955 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5c4510a-0d58-4dcb-924d-47a74eaaa9b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnq4d\" (UID: \"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3\") " pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.709067 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.708976 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab14650a-fd9f-4605-ac6c-7435637be8e7-registry-tls\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.709067 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.708982 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bf0c340-1448-46eb-aa8a-a8c1c9738ba7-metrics-tls\") pod \"dns-default-sgzm5\" (UID: \"9bf0c340-1448-46eb-aa8a-a8c1c9738ba7\") " pod="openshift-dns/dns-default-sgzm5" Jul 07 06:32:45.709067 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.708994 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab14650a-fd9f-4605-ac6c-7435637be8e7-image-registry-private-configuration\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.709067 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.709050 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab14650a-fd9f-4605-ac6c-7435637be8e7-installation-pull-secrets\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.719268 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.719241 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfqjx\" (UniqueName: \"kubernetes.io/projected/9bf0c340-1448-46eb-aa8a-a8c1c9738ba7-kube-api-access-xfqjx\") pod \"dns-default-sgzm5\" (UID: \"9bf0c340-1448-46eb-aa8a-a8c1c9738ba7\") " pod="openshift-dns/dns-default-sgzm5" Jul 07 06:32:45.719603 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.719587 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab14650a-fd9f-4605-ac6c-7435637be8e7-bound-sa-token\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.719660 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.719588 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqbqs\" (UniqueName: \"kubernetes.io/projected/f5c4510a-0d58-4dcb-924d-47a74eaaa9b3-kube-api-access-kqbqs\") pod \"insights-runtime-extractor-hnq4d\" (UID: \"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3\") " pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.719798 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.719783 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npbh5\" (UniqueName: \"kubernetes.io/projected/ab14650a-fd9f-4605-ac6c-7435637be8e7-kube-api-access-npbh5\") pod \"image-registry-5658684c68-khz7t\" (UID: \"ab14650a-fd9f-4605-ac6c-7435637be8e7\") " pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.786928 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.786889 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:45.801062 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.801033 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sgzm5" Jul 07 06:32:45.806265 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.806246 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8z4q\" (UniqueName: \"kubernetes.io/projected/88f4bcf7-85d5-4270-86fd-576d5c2414c8-kube-api-access-w8z4q\") pod \"ingress-canary-vlbw7\" (UID: \"88f4bcf7-85d5-4270-86fd-576d5c2414c8\") " pod="openshift-ingress-canary/ingress-canary-vlbw7" Jul 07 06:32:45.806348 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.806329 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88f4bcf7-85d5-4270-86fd-576d5c2414c8-cert\") pod \"ingress-canary-vlbw7\" (UID: \"88f4bcf7-85d5-4270-86fd-576d5c2414c8\") " pod="openshift-ingress-canary/ingress-canary-vlbw7" Jul 07 06:32:45.808559 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.808538 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88f4bcf7-85d5-4270-86fd-576d5c2414c8-cert\") pod \"ingress-canary-vlbw7\" (UID: \"88f4bcf7-85d5-4270-86fd-576d5c2414c8\") " pod="openshift-ingress-canary/ingress-canary-vlbw7" Jul 07 06:32:45.815089 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.815069 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8z4q\" (UniqueName: \"kubernetes.io/projected/88f4bcf7-85d5-4270-86fd-576d5c2414c8-kube-api-access-w8z4q\") pod \"ingress-canary-vlbw7\" (UID: \"88f4bcf7-85d5-4270-86fd-576d5c2414c8\") " pod="openshift-ingress-canary/ingress-canary-vlbw7" Jul 07 06:32:45.820931 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.820912 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hnq4d" Jul 07 06:32:45.920298 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.920119 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vlbw7" Jul 07 06:32:45.963762 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:45.963727 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9mqv" event={"ID":"acb18573-2d7c-4e7b-9429-dcad9be2ced6","Type":"ContainerStarted","Data":"ee39f40e9457a5b0c6b2832b17de1037556b523fa53b089d0def189838ae4e60"} Jul 07 06:32:46.000911 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.000230 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-f9mqv" podStartSLOduration=8.733887638 podStartE2EDuration="41.000207345s" totalImagesPullingTime="32.266319707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:32:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:32:45.992550464 +0000 UTC m=+41.832391443" watchObservedRunningTime="2026-07-07 06:32:46.000207345 +0000 UTC m=+41.840048318" Jul 07 06:32:46.001293 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.001263 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5658684c68-khz7t"] Jul 07 06:32:46.007422 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:46.006798 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab14650a_fd9f_4605_ac6c_7435637be8e7.slice/crio-7a17027378cf3ac9c22d0cf4f134b43b3b60e73dd21c54e8a7343585769accac WatchSource:0}: Error finding container 7a17027378cf3ac9c22d0cf4f134b43b3b60e73dd21c54e8a7343585769accac: Status 404 returned error can't find the container with id 7a17027378cf3ac9c22d0cf4f134b43b3b60e73dd21c54e8a7343585769accac Jul 07 06:32:46.017180 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.017039 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sgzm5"] Jul 07 06:32:46.041317 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.040536 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hnq4d"] Jul 07 06:32:46.042554 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:46.042516 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5c4510a_0d58_4dcb_924d_47a74eaaa9b3.slice/crio-bcf9c8705cc090604cc294d4fe51c562a290c50862277c69351ccac069a3cafa WatchSource:0}: Error finding container bcf9c8705cc090604cc294d4fe51c562a290c50862277c69351ccac069a3cafa: Status 404 returned error can't find the container with id bcf9c8705cc090604cc294d4fe51c562a290c50862277c69351ccac069a3cafa Jul 07 06:32:46.102247 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.102220 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vlbw7"] Jul 07 06:32:46.105082 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:32:46.105059 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f4bcf7_85d5_4270_86fd_576d5c2414c8.slice/crio-9fa272183508b933e2c6fb5186eb57d16a21fcaaf59abed974e3ccdba89bdb03 WatchSource:0}: Error finding container 9fa272183508b933e2c6fb5186eb57d16a21fcaaf59abed974e3ccdba89bdb03: Status 404 returned error can't find the container with id 9fa272183508b933e2c6fb5186eb57d16a21fcaaf59abed974e3ccdba89bdb03 Jul 07 06:32:46.106383 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.106345 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:46.169351 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.169317 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:46.777149 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.777100 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:32:46.778385 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.777100 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:32:46.778385 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.777784 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:32:46.782279 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.781789 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 06:32:46.782279 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.781842 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dsscv\"" Jul 07 06:32:46.782279 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.781870 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 06:32:46.782279 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.781789 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5wsxr\"" Jul 07 06:32:46.782279 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.782085 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 06:32:46.782279 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.782143 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 06:32:46.969112 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.969031 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnq4d" event={"ID":"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3","Type":"ContainerStarted","Data":"bcf9c8705cc090604cc294d4fe51c562a290c50862277c69351ccac069a3cafa"} Jul 07 06:32:46.971053 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.971022 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vlbw7" event={"ID":"88f4bcf7-85d5-4270-86fd-576d5c2414c8","Type":"ContainerStarted","Data":"9fa272183508b933e2c6fb5186eb57d16a21fcaaf59abed974e3ccdba89bdb03"} Jul 07 06:32:46.972376 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.972338 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sgzm5" event={"ID":"9bf0c340-1448-46eb-aa8a-a8c1c9738ba7","Type":"ContainerStarted","Data":"92d5ef3a4b6a8a8a3fe5efc496637c26cc3cffe43fcbe2bd24f10cedd48b1092"} Jul 07 06:32:46.974921 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.974896 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5658684c68-khz7t" event={"ID":"ab14650a-fd9f-4605-ac6c-7435637be8e7","Type":"ContainerStarted","Data":"0cbef86d987bcf1053ef384a3e48a3697d2b525b5efafbeab9a6034ba6f68ca2"} Jul 07 06:32:46.975006 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.974939 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5658684c68-khz7t" event={"ID":"ab14650a-fd9f-4605-ac6c-7435637be8e7","Type":"ContainerStarted","Data":"7a17027378cf3ac9c22d0cf4f134b43b3b60e73dd21c54e8a7343585769accac"} Jul 07 06:32:46.975006 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.974996 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:32:46.997459 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:46.996572 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5658684c68-khz7t" podStartSLOduration=7.996552664 podStartE2EDuration="7.996552664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:32:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:32:46.996292851 +0000 UTC m=+42.836133818" watchObservedRunningTime="2026-07-07 06:32:46.996552664 +0000 UTC m=+42.836393634" Jul 07 06:32:51.557956 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:51.557913 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:51.657424 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:51.657354 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:51.755393 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:51.755350 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:52.195676 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:52.195654 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:52.301277 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:52.301247 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:52.474816 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:52.474786 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:52.995430 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:52.995393 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnq4d" event={"ID":"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3","Type":"ContainerStarted","Data":"9aa9208fc63b88f1014769f9ff085c11c6bee4d2bf2bd8f8bb4f1c9c648446b5"} Jul 07 06:32:52.996664 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:52.996637 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vlbw7" event={"ID":"88f4bcf7-85d5-4270-86fd-576d5c2414c8","Type":"ContainerStarted","Data":"675a16b5e26dd2192fe0821f37960b9a9c3c46541089541fb67c2fc9c055509a"} Jul 07 06:32:52.998267 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:52.998244 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sgzm5" event={"ID":"9bf0c340-1448-46eb-aa8a-a8c1c9738ba7","Type":"ContainerStarted","Data":"d67d550acce84f2049c8a356afc434a074da5b05ac436ec07ccac0f482a64500"} Jul 07 06:32:52.998357 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:52.998290 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sgzm5" event={"ID":"9bf0c340-1448-46eb-aa8a-a8c1c9738ba7","Type":"ContainerStarted","Data":"4ef1a1b68330b196955637a1eb9960f4aece18d7adf1ff6ebab5e113df8111d0"} Jul 07 06:32:52.998432 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:52.998418 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sgzm5" Jul 07 06:32:53.013381 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:53.013329 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vlbw7" podStartSLOduration=2.022268469 podStartE2EDuration="8.013316144s" totalImagesPullingTime="5.991047675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:32:53.01225932 +0000 UTC m=+48.852100288" watchObservedRunningTime="2026-07-07 06:32:53.013316144 +0000 UTC m=+48.853157102" Jul 07 06:32:53.030933 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:53.030874 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-sgzm5" podStartSLOduration=1.980768954 podStartE2EDuration="8.03085168s" totalImagesPullingTime="6.050082726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:32:53.030279823 +0000 UTC m=+48.870120790" watchObservedRunningTime="2026-07-07 06:32:53.03085168 +0000 UTC m=+48.870692648" Jul 07 06:32:53.354710 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:53.354671 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:53.409680 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:53.409647 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:53.498695 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:53.498663 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:53.602932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:53.602909 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:54.003306 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:54.003216 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnq4d" event={"ID":"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3","Type":"ContainerStarted","Data":"e854b2e465b441abcc12f02e34c806d720aebb173f9e5e62118176c9ddc694c1"} Jul 07 06:32:55.738819 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:55.738784 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:55.815576 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:55.815545 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:55.897701 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:55.897669 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:57.015127 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:57.015095 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnq4d" event={"ID":"f5c4510a-0d58-4dcb-924d-47a74eaaa9b3","Type":"ContainerStarted","Data":"980c07252cb4081535be41820b5bfe4e3afffe0d353b089e867292f251e175a0"} Jul 07 06:32:58.037900 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:58.037852 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hnq4d" podStartSLOduration=2.420406353 podStartE2EDuration="13.037836827s" totalImagesPullingTime="10.617430474s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:32:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:32:58.036164654 +0000 UTC m=+53.876005623" watchObservedRunningTime="2026-07-07 06:32:58.037836827 +0000 UTC m=+53.877677798" Jul 07 06:32:59.300786 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:59.300757 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:59.424097 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:59.424063 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:59.584868 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:59.584787 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:59.647447 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:59.647421 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:32:59.701019 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:32:59.700990 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:00.878816 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:00.878788 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:00.990336 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:00.990303 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:01.095058 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.095028 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:01.127474 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.127444 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj"] Jul 07 06:33:01.130456 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.130397 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.133653 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.133626 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 06:33:01.133771 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.133671 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 06:33:01.133771 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.133748 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 06:33:01.134708 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.134687 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 06:33:01.134819 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.134747 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 06:33:01.134888 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.134827 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 06:33:01.134888 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.134866 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-tk2ww\"" Jul 07 06:33:01.142388 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.142349 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj"] Jul 07 06:33:01.161275 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.161247 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cfn7q"] Jul 07 06:33:01.163471 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.163454 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.166229 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.166207 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 06:33:01.166343 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.166332 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 06:33:01.166425 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.166345 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8tnd9\"" Jul 07 06:33:01.166574 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.166556 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 06:33:01.192480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.192453 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:01.272681 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.272651 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:01.327633 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.327607 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-accelerators-collector-config\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.327758 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.327637 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-sys\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.327758 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.327654 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-textfile\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.327758 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.327674 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.327758 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.327725 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-root\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.327971 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.327757 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r298t\" (UniqueName: \"kubernetes.io/projected/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-kube-api-access-r298t\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.327971 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.327790 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-metrics-client-ca\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.327971 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.327823 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psrqw\" (UniqueName: \"kubernetes.io/projected/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-kube-api-access-psrqw\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.327971 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.327849 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-wtmp\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.327971 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.327885 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.327971 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.327915 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.327971 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.327958 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.328212 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.327984 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.328212 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.328049 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-tls\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.328212 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.328076 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.429088 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429005 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-tls\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.429088 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429042 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.429088 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429079 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-accelerators-collector-config\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.429343 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429110 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-sys\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.429343 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429136 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-textfile\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.429343 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:33:01.429151 2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 06:33:01.429343 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429164 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.429343 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429189 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-root\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.429343 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:33:01.429210 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-tls podName:f6837d75-a0c8-4dee-a2b1-f5844de3f0e8 nodeName:}" failed. No retries permitted until 2026-07-07 06:33:01.92919207 +0000 UTC m=+57.769033019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-tls") pod "node-exporter-cfn7q" (UID: "f6837d75-a0c8-4dee-a2b1-f5844de3f0e8") : secret "node-exporter-tls" not found Jul 07 06:33:01.429343 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429229 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-sys\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.429343 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429325 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-root\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.429343 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429237 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r298t\" (UniqueName: \"kubernetes.io/projected/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-kube-api-access-r298t\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.429826 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429405 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-metrics-client-ca\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.429826 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429438 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psrqw\" (UniqueName: \"kubernetes.io/projected/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-kube-api-access-psrqw\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.429826 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429463 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-wtmp\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.429826 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429494 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.429826 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429526 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.429826 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429565 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.429826 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429570 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-textfile\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.429826 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429592 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.429826 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429729 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-wtmp\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.429826 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:33:01.429821 2579 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 07 06:33:01.430194 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:33:01.429877 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-kube-state-metrics-tls podName:5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe nodeName:}" failed. No retries permitted until 2026-07-07 06:33:01.929862067 +0000 UTC m=+57.769703029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-g7lvj" (UID: "5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe") : secret "kube-state-metrics-tls" not found Jul 07 06:33:01.430194 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429875 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.430194 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429873 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-accelerators-collector-config\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.430194 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.429937 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.430194 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.430012 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.430429 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.430332 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-metrics-client-ca\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.431824 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.431809 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.432050 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.432034 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.440954 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.440930 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r298t\" (UniqueName: \"kubernetes.io/projected/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-kube-api-access-r298t\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.441461 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.441445 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psrqw\" (UniqueName: \"kubernetes.io/projected/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-kube-api-access-psrqw\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.489869 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.489833 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:01.613100 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.613066 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:01.715461 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.715385 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:01.816653 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.816623 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:01.932859 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.932831 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:01.933182 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.932881 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-tls\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.935086 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.935066 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f6837d75-a0c8-4dee-a2b1-f5844de3f0e8-node-exporter-tls\") pod \"node-exporter-cfn7q\" (UID: \"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8\") " pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:01.935382 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:01.935348 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-g7lvj\" (UID: \"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:02.039311 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.039280 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" Jul 07 06:33:02.053338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.053311 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:02.072050 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.072026 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cfn7q" Jul 07 06:33:02.081132 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:33:02.081090 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6837d75_a0c8_4dee_a2b1_f5844de3f0e8.slice/crio-dfc93b7613e6d4a24be3f1070dd9023e32d841a3af6a8216b925be7585139bc6 WatchSource:0}: Error finding container dfc93b7613e6d4a24be3f1070dd9023e32d841a3af6a8216b925be7585139bc6: Status 404 returned error can't find the container with id dfc93b7613e6d4a24be3f1070dd9023e32d841a3af6a8216b925be7585139bc6 Jul 07 06:33:02.155312 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.155277 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:02.167494 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.167391 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj"] Jul 07 06:33:02.169799 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:33:02.169776 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e3ca3ab_5ff9_4f9d_b5b2_646ee1219efe.slice/crio-0995e7fd239083c8db2628ef72a6b67c2b825ed984684e2527b2c47164801a1e WatchSource:0}: Error finding container 0995e7fd239083c8db2628ef72a6b67c2b825ed984684e2527b2c47164801a1e: Status 404 returned error can't find the container with id 0995e7fd239083c8db2628ef72a6b67c2b825ed984684e2527b2c47164801a1e Jul 07 06:33:02.204689 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.204668 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 06:33:02.207357 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.207344 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.209950 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.209930 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 06:33:02.210156 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.210140 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 06:33:02.210274 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.210174 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 06:33:02.210274 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.210214 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 06:33:02.210425 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.210335 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 06:33:02.210425 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.210348 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 06:33:02.210425 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.210354 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-srxw8\"" Jul 07 06:33:02.210614 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.210597 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 06:33:02.210670 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.210601 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 06:33:02.210848 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.210832 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 06:33:02.224587 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.224552 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 06:33:02.230008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.229989 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:02.234539 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.234524 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wht5\" (UniqueName: \"kubernetes.io/projected/f740c654-b39f-4226-acf0-bcd17a3f7b40-kube-api-access-5wht5\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.234586 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.234547 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.234586 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.234566 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-config-volume\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.234698 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.234587 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.234698 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.234608 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.234698 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.234684 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f740c654-b39f-4226-acf0-bcd17a3f7b40-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.234837 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.234728 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.234837 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.234754 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f740c654-b39f-4226-acf0-bcd17a3f7b40-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.234837 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.234786 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f740c654-b39f-4226-acf0-bcd17a3f7b40-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.234837 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.234821 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-web-config\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.234977 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.234887 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.234977 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.234903 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f740c654-b39f-4226-acf0-bcd17a3f7b40-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.234977 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.234921 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f740c654-b39f-4226-acf0-bcd17a3f7b40-config-out\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.335593 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.335508 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-config-volume\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.335593 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.335553 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.335796 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:33:02.335652 2579 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 07 06:33:02.335796 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.335670 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.335796 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:33:02.335714 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-main-tls podName:f740c654-b39f-4226-acf0-bcd17a3f7b40 nodeName:}" failed. No retries permitted until 2026-07-07 06:33:02.835693836 +0000 UTC m=+58.675534807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "f740c654-b39f-4226-acf0-bcd17a3f7b40") : secret "alertmanager-main-tls" not found Jul 07 06:33:02.335796 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.335745 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f740c654-b39f-4226-acf0-bcd17a3f7b40-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.335796 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.335783 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.336042 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.335809 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f740c654-b39f-4226-acf0-bcd17a3f7b40-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.336042 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.335843 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f740c654-b39f-4226-acf0-bcd17a3f7b40-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.336042 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.335871 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-web-config\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.336042 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.335992 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.336042 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.336018 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f740c654-b39f-4226-acf0-bcd17a3f7b40-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.336288 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.336048 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f740c654-b39f-4226-acf0-bcd17a3f7b40-config-out\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.336288 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.336088 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wht5\" (UniqueName: \"kubernetes.io/projected/f740c654-b39f-4226-acf0-bcd17a3f7b40-kube-api-access-5wht5\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.336288 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.336114 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.336468 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:33:02.336301 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f740c654-b39f-4226-acf0-bcd17a3f7b40-alertmanager-trusted-ca-bundle podName:f740c654-b39f-4226-acf0-bcd17a3f7b40 nodeName:}" failed. No retries permitted until 2026-07-07 06:33:02.836282484 +0000 UTC m=+58.676123447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/f740c654-b39f-4226-acf0-bcd17a3f7b40-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "f740c654-b39f-4226-acf0-bcd17a3f7b40") : configmap references non-existent config key: ca-bundle.crt Jul 07 06:33:02.337050 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.336544 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f740c654-b39f-4226-acf0-bcd17a3f7b40-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.337168 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.336623 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f740c654-b39f-4226-acf0-bcd17a3f7b40-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.338577 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.338554 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-config-volume\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.338675 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.338583 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-web-config\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.338675 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.338554 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.338924 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.338902 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f740c654-b39f-4226-acf0-bcd17a3f7b40-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.338958 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.338941 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.339003 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.338978 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.339147 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.339132 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f740c654-b39f-4226-acf0-bcd17a3f7b40-config-out\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.339672 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.339657 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.347167 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.347151 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wht5\" (UniqueName: \"kubernetes.io/projected/f740c654-b39f-4226-acf0-bcd17a3f7b40-kube-api-access-5wht5\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.839184 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.839149 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.839404 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.839205 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f740c654-b39f-4226-acf0-bcd17a3f7b40-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:02.839404 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:33:02.839312 2579 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 07 06:33:02.839404 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:33:02.839396 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-main-tls podName:f740c654-b39f-4226-acf0-bcd17a3f7b40 nodeName:}" failed. No retries permitted until 2026-07-07 06:33:03.839377066 +0000 UTC m=+59.679218025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "f740c654-b39f-4226-acf0-bcd17a3f7b40") : secret "alertmanager-main-tls" not found Jul 07 06:33:02.840147 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:02.840122 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f740c654-b39f-4226-acf0-bcd17a3f7b40-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:03.005758 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.005724 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sgzm5" Jul 07 06:33:03.035888 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.035846 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cfn7q" event={"ID":"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8","Type":"ContainerStarted","Data":"dfc93b7613e6d4a24be3f1070dd9023e32d841a3af6a8216b925be7585139bc6"} Jul 07 06:33:03.037447 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.037397 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" event={"ID":"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe","Type":"ContainerStarted","Data":"0995e7fd239083c8db2628ef72a6b67c2b825ed984684e2527b2c47164801a1e"} Jul 07 06:33:03.161260 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.161018 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6f6bb9757c-s52tl"] Jul 07 06:33:03.163733 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.163712 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.166413 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.166251 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 06:33:03.166413 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.166254 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-grq9q\"" Jul 07 06:33:03.166413 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.166290 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 06:33:03.166735 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.166714 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 06:33:03.166816 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.166748 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-esklcb2av075i\"" Jul 07 06:33:03.166816 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.166770 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 06:33:03.166920 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.166721 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 06:33:03.178285 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.178265 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6f6bb9757c-s52tl"] Jul 07 06:33:03.244761 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.244723 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.244898 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.244767 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.244898 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.244853 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnvfm\" (UniqueName: \"kubernetes.io/projected/5c3bb6d4-279f-4c40-af27-053f8f50b538-kube-api-access-vnvfm\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.245017 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.244900 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.245017 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.244972 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c3bb6d4-279f-4c40-af27-053f8f50b538-metrics-client-ca\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.245017 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.244993 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-grpc-tls\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.245121 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.245080 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-thanos-querier-tls\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.245208 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.245189 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.329727 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.329687 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:03.346382 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.346343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.346495 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.346416 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.346495 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.346450 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.346495 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.346485 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnvfm\" (UniqueName: \"kubernetes.io/projected/5c3bb6d4-279f-4c40-af27-053f8f50b538-kube-api-access-vnvfm\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.346618 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.346512 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.346751 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.346721 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c3bb6d4-279f-4c40-af27-053f8f50b538-metrics-client-ca\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.346820 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.346771 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-grpc-tls\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.346875 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.346818 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-thanos-querier-tls\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.347598 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.347519 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c3bb6d4-279f-4c40-af27-053f8f50b538-metrics-client-ca\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.349300 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.349253 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.349419 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.349329 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.349419 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.349410 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.349522 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.349443 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.349810 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.349792 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-grpc-tls\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.350318 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.350295 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5c3bb6d4-279f-4c40-af27-053f8f50b538-secret-thanos-querier-tls\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.355235 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.355214 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnvfm\" (UniqueName: \"kubernetes.io/projected/5c3bb6d4-279f-4c40-af27-053f8f50b538-kube-api-access-vnvfm\") pod \"thanos-querier-6f6bb9757c-s52tl\" (UID: \"5c3bb6d4-279f-4c40-af27-053f8f50b538\") " pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.475340 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.475317 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:03.532884 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.532852 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:03.612140 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.611957 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6f6bb9757c-s52tl"] Jul 07 06:33:03.638857 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.638826 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:03.819474 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:33:03.819445 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c3bb6d4_279f_4c40_af27_053f8f50b538.slice/crio-9b2e257f625a827ce09722ff6cde9888463e4814b791672c2b8e7748fa823c7f WatchSource:0}: Error finding container 9b2e257f625a827ce09722ff6cde9888463e4814b791672c2b8e7748fa823c7f: Status 404 returned error can't find the container with id 9b2e257f625a827ce09722ff6cde9888463e4814b791672c2b8e7748fa823c7f Jul 07 06:33:03.850582 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.850559 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:03.852989 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:03.852972 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f740c654-b39f-4226-acf0-bcd17a3f7b40-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f740c654-b39f-4226-acf0-bcd17a3f7b40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:04.016653 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:04.016623 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 06:33:04.044051 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:04.043689 2579 generic.go:356] "Generic (PLEG): container finished" podID="f6837d75-a0c8-4dee-a2b1-f5844de3f0e8" containerID="3c09ec1bdb8786a3cf9909d4218ac2f0fecb1801ab8b64fab0583ee9ad480632" exitCode=0 Jul 07 06:33:04.044051 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:04.043800 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cfn7q" event={"ID":"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8","Type":"ContainerDied","Data":"3c09ec1bdb8786a3cf9909d4218ac2f0fecb1801ab8b64fab0583ee9ad480632"} Jul 07 06:33:04.052495 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:04.048967 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" event={"ID":"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe","Type":"ContainerStarted","Data":"02c99407496a8a342e32442ab6025f70028b4f18adbed347a1602f30dca840ef"} Jul 07 06:33:04.052495 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:04.049029 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" event={"ID":"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe","Type":"ContainerStarted","Data":"1ec924f3bfe9ada5313eac2615eef98b3a645814626acae5ab5a9529d9f071d3"} Jul 07 06:33:04.052495 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:04.049056 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" event={"ID":"5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe","Type":"ContainerStarted","Data":"d628b0a0a5c9f9b1ecc1f3db39ca82fb3f4e7c550d91b4872ea35fdb0aa5c310"} Jul 07 06:33:04.055993 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:04.054061 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" event={"ID":"5c3bb6d4-279f-4c40-af27-053f8f50b538","Type":"ContainerStarted","Data":"9b2e257f625a827ce09722ff6cde9888463e4814b791672c2b8e7748fa823c7f"} Jul 07 06:33:04.096473 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:04.096422 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g7lvj" podStartSLOduration=1.427056284 podStartE2EDuration="3.09640052s" totalImagesPullingTime="1.669344236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:33:04.095994795 +0000 UTC m=+59.935835773" watchObservedRunningTime="2026-07-07 06:33:04.09640052 +0000 UTC m=+59.936241489" Jul 07 06:33:04.096764 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:04.096738 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:04.159795 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:04.159768 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:04.172804 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:04.170988 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 06:33:04.434274 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:04.434199 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:04.535051 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:04.535017 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:05.059044 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:05.059009 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f740c654-b39f-4226-acf0-bcd17a3f7b40","Type":"ContainerStarted","Data":"e95ff4b60d22c274733d805407072440800843850ad616c600248e725885dddc"} Jul 07 06:33:05.061463 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:05.061427 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cfn7q" event={"ID":"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8","Type":"ContainerStarted","Data":"8c7934014005bd56ae7a42bc9edf5acc729357cfede1d73625d13535d81f5fc8"} Jul 07 06:33:05.061599 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:05.061496 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cfn7q" event={"ID":"f6837d75-a0c8-4dee-a2b1-f5844de3f0e8","Type":"ContainerStarted","Data":"0930972cc77778cc9a92553307f0534dc452c4211d4a495ae2b430cd64f2188c"} Jul 07 06:33:05.081450 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:05.081407 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cfn7q" podStartSLOduration=2.800712639 podStartE2EDuration="4.081393113s" totalImagesPullingTime="1.216196388s" totalInitContainerRuntime="64.484086ms" isStatefulPod=true podCreationTimestamp="2026-07-07 06:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:33:05.0801778 +0000 UTC m=+60.920018772" watchObservedRunningTime="2026-07-07 06:33:05.081393113 +0000 UTC m=+60.921234079" Jul 07 06:33:05.742579 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:05.742548 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:05.791740 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:05.791710 2579 patch_prober.go:28] interesting pod/image-registry-5658684c68-khz7t container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 07 06:33:05.791901 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:05.791758 2579 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5658684c68-khz7t" podUID="ab14650a-fd9f-4605-ac6c-7435637be8e7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 06:33:05.846775 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:05.846740 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:05.937748 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:05.937713 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:05.945024 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:05.945000 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vn8hx" Jul 07 06:33:06.087577 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.087544 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:06.186427 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.186393 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:06.249767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.249731 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:06.380854 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.380836 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82"] Jul 07 06:33:06.383604 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.383585 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.386245 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.386226 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 06:33:06.386358 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.386289 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 06:33:06.386358 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.386316 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-lrxdh\"" Jul 07 06:33:06.386498 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.386483 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 06:33:06.386567 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.386555 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 06:33:06.386702 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.386691 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 06:33:06.392162 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.392142 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 06:33:06.402947 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.402926 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82"] Jul 07 06:33:06.472542 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.472511 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-metrics-client-ca\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.472638 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.472545 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.472638 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.472576 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-federate-client-tls\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.472638 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.472598 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.472638 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.472636 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-secret-telemeter-client\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.472814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.472674 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-serving-certs-ca-bundle\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.472814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.472704 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqmpj\" (UniqueName: \"kubernetes.io/projected/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-kube-api-access-pqmpj\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.472814 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.472760 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-telemeter-client-tls\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.574028 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.573987 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-serving-certs-ca-bundle\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.574160 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.574069 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqmpj\" (UniqueName: \"kubernetes.io/projected/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-kube-api-access-pqmpj\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.574452 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.574428 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-telemeter-client-tls\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.574578 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.574525 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-metrics-client-ca\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.574578 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.574560 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.574691 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.574620 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-federate-client-tls\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.574691 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.574643 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.574792 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.574690 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-secret-telemeter-client\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.574792 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.574690 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-serving-certs-ca-bundle\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.575192 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.575169 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-metrics-client-ca\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.575714 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.575682 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.580594 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.580557 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.580705 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.580644 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-telemeter-client-tls\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.580705 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.580678 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-secret-telemeter-client\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.581339 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.581313 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-federate-client-tls\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.583655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.583638 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqmpj\" (UniqueName: \"kubernetes.io/projected/c8e9b298-5749-45ad-afd4-b7de45d3a2e6-kube-api-access-pqmpj\") pod \"telemeter-client-8fbcf4c6d-xtd82\" (UID: \"c8e9b298-5749-45ad-afd4-b7de45d3a2e6\") " pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.761780 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.761751 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" Jul 07 06:33:06.883846 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:06.883819 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82"] Jul 07 06:33:06.886019 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:33:06.885995 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e9b298_5749_45ad_afd4_b7de45d3a2e6.slice/crio-ee66f6b45ad35f13a7744e4f3ec5d2ff19f25c71ce5a21dcecadecace12b82e6 WatchSource:0}: Error finding container ee66f6b45ad35f13a7744e4f3ec5d2ff19f25c71ce5a21dcecadecace12b82e6: Status 404 returned error can't find the container with id ee66f6b45ad35f13a7744e4f3ec5d2ff19f25c71ce5a21dcecadecace12b82e6 Jul 07 06:33:07.074433 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:07.074391 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" event={"ID":"5c3bb6d4-279f-4c40-af27-053f8f50b538","Type":"ContainerStarted","Data":"2585959adb569bb854ffdc0d1cf4759022d73ac63dc47e26913ff8f29bca4e6e"} Jul 07 06:33:07.074611 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:07.074481 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" event={"ID":"5c3bb6d4-279f-4c40-af27-053f8f50b538","Type":"ContainerStarted","Data":"e1cbe653e63ddc275a8ed79ccae0e942bd3cf2a3bcc864b3c4d4dd7a0dd30149"} Jul 07 06:33:07.074611 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:07.074513 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" event={"ID":"5c3bb6d4-279f-4c40-af27-053f8f50b538","Type":"ContainerStarted","Data":"888029b17fa8181b95df64dd4409ea4c345cd69d80296ee929392e5a45ce0848"} Jul 07 06:33:07.075429 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:07.075396 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" event={"ID":"c8e9b298-5749-45ad-afd4-b7de45d3a2e6","Type":"ContainerStarted","Data":"ee66f6b45ad35f13a7744e4f3ec5d2ff19f25c71ce5a21dcecadecace12b82e6"} Jul 07 06:33:07.076751 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:07.076725 2579 generic.go:356] "Generic (PLEG): container finished" podID="f740c654-b39f-4226-acf0-bcd17a3f7b40" containerID="6f839820770a359c9e4de4c49b6fa423a700aa7b2eb68a336efd085e4446c53d" exitCode=0 Jul 07 06:33:07.076857 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:07.076762 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f740c654-b39f-4226-acf0-bcd17a3f7b40","Type":"ContainerDied","Data":"6f839820770a359c9e4de4c49b6fa423a700aa7b2eb68a336efd085e4446c53d"} Jul 07 06:33:07.410061 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:07.410028 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:07.985748 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:07.985676 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5658684c68-khz7t" Jul 07 06:33:08.084432 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:08.084400 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" event={"ID":"5c3bb6d4-279f-4c40-af27-053f8f50b538","Type":"ContainerStarted","Data":"7d3453813fd21a37f95c55cf9bce3e261f39324ab1590192c68841fb190bd99c"} Jul 07 06:33:08.084591 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:08.084461 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" event={"ID":"5c3bb6d4-279f-4c40-af27-053f8f50b538","Type":"ContainerStarted","Data":"8207e2dbc4fb3a4ef4089b5855e856fd1f9a59d840b91d6c95aecd0157b1f4ff"} Jul 07 06:33:08.084591 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:08.084486 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" event={"ID":"5c3bb6d4-279f-4c40-af27-053f8f50b538","Type":"ContainerStarted","Data":"89fe8e5acfcf3d2aa87517823a45e9eb415d35cbfa8324d9d59ce313adee237c"} Jul 07 06:33:08.084591 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:08.084574 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:08.108731 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:08.108682 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" podStartSLOduration=1.647818446 podStartE2EDuration="5.108666988s" totalImagesPullingTime="3.460848542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:33:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:33:08.107122134 +0000 UTC m=+63.946963126" watchObservedRunningTime="2026-07-07 06:33:08.108666988 +0000 UTC m=+63.948507956" Jul 07 06:33:08.159499 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:08.159473 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:08.259332 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:08.259251 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:08.359280 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:08.359246 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:10.092962 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.092864 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" event={"ID":"c8e9b298-5749-45ad-afd4-b7de45d3a2e6","Type":"ContainerStarted","Data":"9c26ddef5884f985d682efefeea4a2f50cbfa623fe4289eac77e3b94222ef851"} Jul 07 06:33:10.093421 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.092972 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" event={"ID":"c8e9b298-5749-45ad-afd4-b7de45d3a2e6","Type":"ContainerStarted","Data":"99f2e3d14638f547fe568afd8040ffd64e256c853a0950c410188e73a2ec0b76"} Jul 07 06:33:10.093421 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.093004 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" event={"ID":"c8e9b298-5749-45ad-afd4-b7de45d3a2e6","Type":"ContainerStarted","Data":"9af1cc02e3fb11d50c99e56c2896f36ad36d9956541b0314987ded362c1a4e51"} Jul 07 06:33:10.095908 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.095882 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f740c654-b39f-4226-acf0-bcd17a3f7b40","Type":"ContainerStarted","Data":"c118caaf72a8930325396a425d9937600bc9b70225a8de18777c2e9690998fc2"} Jul 07 06:33:10.096033 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.095947 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f740c654-b39f-4226-acf0-bcd17a3f7b40","Type":"ContainerStarted","Data":"c110a472b6fc833c35c64752dc8edb04e85bbd4d1a7e25c7465a44c37da1fb93"} Jul 07 06:33:10.096033 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.095988 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f740c654-b39f-4226-acf0-bcd17a3f7b40","Type":"ContainerStarted","Data":"1febc9203eb3d75ff6d3465fed5c5def08a3396732755935d65ae41f50f61021"} Jul 07 06:33:10.096152 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.096034 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f740c654-b39f-4226-acf0-bcd17a3f7b40","Type":"ContainerStarted","Data":"bb72a039046d70b968a3b660f05d1e300f351884d41826dcd90765354ec70669"} Jul 07 06:33:10.096152 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.096073 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f740c654-b39f-4226-acf0-bcd17a3f7b40","Type":"ContainerStarted","Data":"ff1ca480551a957915fdfd694f293e5f2126ed288cad1791e24765bcb4454146"} Jul 07 06:33:10.096152 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.096116 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f740c654-b39f-4226-acf0-bcd17a3f7b40","Type":"ContainerStarted","Data":"23bcf943c16229c48a0014f3c650ea980bdf0b65149bf2fbe5ad734ae5035555"} Jul 07 06:33:10.116229 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.116181 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-8fbcf4c6d-xtd82" podStartSLOduration=1.467087509 podStartE2EDuration="4.116167039s" totalImagesPullingTime="2.64907953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:33:10.113502455 +0000 UTC m=+65.953343435" watchObservedRunningTime="2026-07-07 06:33:10.116167039 +0000 UTC m=+65.956008006" Jul 07 06:33:10.141543 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.141499 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.5036303970000002 podStartE2EDuration="8.141487276s" totalImagesPullingTime="4.618288864s" totalInitContainerRuntime="19.568015ms" isStatefulPod=false podCreationTimestamp="2026-07-07 06:33:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:33:10.13933195 +0000 UTC m=+65.979172916" watchObservedRunningTime="2026-07-07 06:33:10.141487276 +0000 UTC m=+65.981328242" Jul 07 06:33:10.265510 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.265474 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:10.365247 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.365179 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:10.408502 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.408473 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:33:10.411657 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.411642 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 06:33:10.421043 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.421018 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/997fb304-b3ae-42e0-8a84-db42daea16b1-original-pull-secret\") pod \"global-pull-secret-syncer-4bdk4\" (UID: \"997fb304-b3ae-42e0-8a84-db42daea16b1\") " pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:33:10.492322 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.492297 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4bdk4" Jul 07 06:33:10.509666 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.509643 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs\") pod \"network-metrics-daemon-thndk\" (UID: \"e3255519-1e37-4781-8d6e-14394758a8b2\") " pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:33:10.509753 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.509693 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsp8h\" (UniqueName: \"kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h\") pod \"network-check-target-mqsn2\" (UID: \"d592a475-9102-419a-a6e0-5877d1838ca9\") " pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:33:10.512568 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.512544 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 06:33:10.512568 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.512562 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 06:33:10.522829 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.522801 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3255519-1e37-4781-8d6e-14394758a8b2-metrics-certs\") pod \"network-metrics-daemon-thndk\" (UID: \"e3255519-1e37-4781-8d6e-14394758a8b2\") " pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:33:10.523405 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.523392 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 06:33:10.533985 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.533962 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsp8h\" (UniqueName: \"kubernetes.io/projected/d592a475-9102-419a-a6e0-5877d1838ca9-kube-api-access-vsp8h\") pod \"network-check-target-mqsn2\" (UID: \"d592a475-9102-419a-a6e0-5877d1838ca9\") " pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:33:10.616992 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.616918 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4bdk4"] Jul 07 06:33:10.619204 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:33:10.619177 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod997fb304_b3ae_42e0_8a84_db42daea16b1.slice/crio-e9ca63afc343855f9da379df09d539b79d1e6d025c4a7d1f1f49139ea6b2487d WatchSource:0}: Error finding container e9ca63afc343855f9da379df09d539b79d1e6d025c4a7d1f1f49139ea6b2487d: Status 404 returned error can't find the container with id e9ca63afc343855f9da379df09d539b79d1e6d025c4a7d1f1f49139ea6b2487d Jul 07 06:33:10.803415 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.803380 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5wsxr\"" Jul 07 06:33:10.811591 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.811567 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-thndk" Jul 07 06:33:10.811924 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.811908 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dsscv\"" Jul 07 06:33:10.818842 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.818825 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:33:10.956756 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.956692 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-thndk"] Jul 07 06:33:10.959575 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:33:10.959527 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3255519_1e37_4781_8d6e_14394758a8b2.slice/crio-5b9fac6a3de153cce71994e79db3e36fc4bf4192293a7dbdb004dc9e19d9fd9d WatchSource:0}: Error finding container 5b9fac6a3de153cce71994e79db3e36fc4bf4192293a7dbdb004dc9e19d9fd9d: Status 404 returned error can't find the container with id 5b9fac6a3de153cce71994e79db3e36fc4bf4192293a7dbdb004dc9e19d9fd9d Jul 07 06:33:10.974948 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:10.974928 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mqsn2"] Jul 07 06:33:10.977737 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:33:10.977715 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd592a475_9102_419a_a6e0_5877d1838ca9.slice/crio-72c37cacb8b25c77cf6db2cf0cbfe574e4e8d8f0c812bf6f38ffde2be3a95cb1 WatchSource:0}: Error finding container 72c37cacb8b25c77cf6db2cf0cbfe574e4e8d8f0c812bf6f38ffde2be3a95cb1: Status 404 returned error can't find the container with id 72c37cacb8b25c77cf6db2cf0cbfe574e4e8d8f0c812bf6f38ffde2be3a95cb1 Jul 07 06:33:11.100597 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:11.100566 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mqsn2" event={"ID":"d592a475-9102-419a-a6e0-5877d1838ca9","Type":"ContainerStarted","Data":"72c37cacb8b25c77cf6db2cf0cbfe574e4e8d8f0c812bf6f38ffde2be3a95cb1"} Jul 07 06:33:11.101714 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:11.101686 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4bdk4" event={"ID":"997fb304-b3ae-42e0-8a84-db42daea16b1","Type":"ContainerStarted","Data":"e9ca63afc343855f9da379df09d539b79d1e6d025c4a7d1f1f49139ea6b2487d"} Jul 07 06:33:11.102841 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:11.102818 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-thndk" event={"ID":"e3255519-1e37-4781-8d6e-14394758a8b2","Type":"ContainerStarted","Data":"5b9fac6a3de153cce71994e79db3e36fc4bf4192293a7dbdb004dc9e19d9fd9d"} Jul 07 06:33:11.139435 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:11.139358 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:11.198166 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:11.198140 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:13.112780 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:13.112738 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:13.113975 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:13.113937 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-thndk" event={"ID":"e3255519-1e37-4781-8d6e-14394758a8b2","Type":"ContainerStarted","Data":"f7339257d0d1899043500ceab774ee86231569fed5701bdcb21c530f0476d92a"} Jul 07 06:33:13.114113 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:13.114019 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-thndk" event={"ID":"e3255519-1e37-4781-8d6e-14394758a8b2","Type":"ContainerStarted","Data":"131241c8851efab5cf14dd5ac9e97f9e02d65172179afe9abe0be9b0a888c8f0"} Jul 07 06:33:13.132978 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:13.132839 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-thndk" podStartSLOduration=66.955908884 podStartE2EDuration="1m8.132818696s" totalImagesPullingTime="1.176909812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:33:13.13067707 +0000 UTC m=+68.970518037" watchObservedRunningTime="2026-07-07 06:33:13.132818696 +0000 UTC m=+68.972659664" Jul 07 06:33:13.210755 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:13.210717 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:13.272043 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:13.271957 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:13.334533 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:13.334460 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:14.095964 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:14.095937 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6f6bb9757c-s52tl" Jul 07 06:33:16.127704 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:16.127670 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mqsn2" event={"ID":"d592a475-9102-419a-a6e0-5877d1838ca9","Type":"ContainerStarted","Data":"43cae490cd2f293697ce72d94468d5124ead6745bfc120072a237113a5618f31"} Jul 07 06:33:16.128112 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:16.127858 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:33:16.129224 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:16.129201 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4bdk4" event={"ID":"997fb304-b3ae-42e0-8a84-db42daea16b1","Type":"ContainerStarted","Data":"9d791eb50cf7b373e0e0371258c016743f1e04d6f75447e01511a8773e94cb25"} Jul 07 06:33:16.149476 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:16.149426 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mqsn2" podStartSLOduration=66.447277766 podStartE2EDuration="1m11.149412228s" totalImagesPullingTime="4.702134462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:33:16.148497056 +0000 UTC m=+71.988338024" watchObservedRunningTime="2026-07-07 06:33:16.149412228 +0000 UTC m=+71.989253194" Jul 07 06:33:16.165128 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:16.165081 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4bdk4" podStartSLOduration=66.100547294 podStartE2EDuration="1m11.165071033s" totalImagesPullingTime="5.064523739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 06:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:33:16.164439125 +0000 UTC m=+72.004280091" watchObservedRunningTime="2026-07-07 06:33:16.165071033 +0000 UTC m=+72.004912015" Jul 07 06:33:16.849325 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:16.849293 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:16.949000 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:16.948970 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:17.779957 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:17.779911 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:17.883337 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:17.883305 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:17.981284 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:17.981251 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:18.070713 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:18.070629 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:18.194489 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:18.194439 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:20.146161 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:20.146133 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:20.243872 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:20.243838 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:21.904671 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:21.904637 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:22.004812 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:22.004780 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:22.061596 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:22.061562 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:25.187253 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:25.187221 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:25.241233 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:25.241203 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:40.190513 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:40.190353 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:40.247607 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:40.247577 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:47.134636 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:47.134605 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mqsn2" Jul 07 06:33:49.233078 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:49.233044 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:33:49.295186 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:33:49.295153 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:34:17.274698 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:34:17.274660 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:34:17.375811 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:34:17.375777 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:34:20.439500 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:34:20.439467 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:34:20.562028 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:34:20.561998 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:34:20.634983 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:34:20.634952 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:34:21.157933 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:34:21.157898 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:34:29.275075 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:34:29.275041 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:34:29.334631 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:34:29.334597 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:34:31.161713 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:34:31.161674 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:34:31.263547 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:34:31.263514 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:34:31.319021 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:34:31.318994 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:34:40.190712 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:34:40.190675 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:34:40.291049 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:34:40.291018 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:08.384001 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:08.383916 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:08.441975 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:08.441947 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:21.056994 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.056943 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:21.150022 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.149991 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:21.779083 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.779048 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q"] Jul 07 06:35:21.782532 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.782508 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" Jul 07 06:35:21.785129 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.785107 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 06:35:21.785252 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.785141 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 06:35:21.785252 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.785143 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bmkmj\"" Jul 07 06:35:21.790813 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.790790 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q"] Jul 07 06:35:21.821318 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.821283 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q\" (UID: \"797e47eb-12e3-4cee-88ce-2bb79f6dbc56\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" Jul 07 06:35:21.821512 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.821332 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4crfm\" (UniqueName: \"kubernetes.io/projected/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-kube-api-access-4crfm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q\" (UID: \"797e47eb-12e3-4cee-88ce-2bb79f6dbc56\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" Jul 07 06:35:21.821512 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.821428 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q\" (UID: \"797e47eb-12e3-4cee-88ce-2bb79f6dbc56\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" Jul 07 06:35:21.922283 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.922244 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q\" (UID: \"797e47eb-12e3-4cee-88ce-2bb79f6dbc56\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" Jul 07 06:35:21.922501 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.922307 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4crfm\" (UniqueName: \"kubernetes.io/projected/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-kube-api-access-4crfm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q\" (UID: \"797e47eb-12e3-4cee-88ce-2bb79f6dbc56\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" Jul 07 06:35:21.922501 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.922399 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q\" (UID: \"797e47eb-12e3-4cee-88ce-2bb79f6dbc56\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" Jul 07 06:35:21.922629 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.922600 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q\" (UID: \"797e47eb-12e3-4cee-88ce-2bb79f6dbc56\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" Jul 07 06:35:21.922745 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.922727 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q\" (UID: \"797e47eb-12e3-4cee-88ce-2bb79f6dbc56\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" Jul 07 06:35:21.930589 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:21.930563 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4crfm\" (UniqueName: \"kubernetes.io/projected/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-kube-api-access-4crfm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q\" (UID: \"797e47eb-12e3-4cee-88ce-2bb79f6dbc56\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" Jul 07 06:35:22.091951 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:22.091849 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" Jul 07 06:35:22.218889 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:22.218864 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q"] Jul 07 06:35:22.221544 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:35:22.221512 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod797e47eb_12e3_4cee_88ce_2bb79f6dbc56.slice/crio-a41955858e4ea438d130cde1798a8debe6eaa3a325b3f6104c5e9135bb4903d9 WatchSource:0}: Error finding container a41955858e4ea438d130cde1798a8debe6eaa3a325b3f6104c5e9135bb4903d9: Status 404 returned error can't find the container with id a41955858e4ea438d130cde1798a8debe6eaa3a325b3f6104c5e9135bb4903d9 Jul 07 06:35:22.575983 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:22.575953 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" event={"ID":"797e47eb-12e3-4cee-88ce-2bb79f6dbc56","Type":"ContainerStarted","Data":"a41955858e4ea438d130cde1798a8debe6eaa3a325b3f6104c5e9135bb4903d9"} Jul 07 06:35:29.603850 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:29.603815 2579 generic.go:356] "Generic (PLEG): container finished" podID="797e47eb-12e3-4cee-88ce-2bb79f6dbc56" containerID="b5bb599b3b015e06e9c74ef58622918031b7cc6d61d40b63fd73053e1f59c1c5" exitCode=0 Jul 07 06:35:29.604196 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:29.603880 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" event={"ID":"797e47eb-12e3-4cee-88ce-2bb79f6dbc56","Type":"ContainerDied","Data":"b5bb599b3b015e06e9c74ef58622918031b7cc6d61d40b63fd73053e1f59c1c5"} Jul 07 06:35:32.618085 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:32.618054 2579 generic.go:356] "Generic (PLEG): container finished" podID="797e47eb-12e3-4cee-88ce-2bb79f6dbc56" containerID="0323e584469a1afeda7dc01c79a16ca10b77ec662b6e45d6b27c20aa88e2bebe" exitCode=0 Jul 07 06:35:32.618485 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:32.618107 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" event={"ID":"797e47eb-12e3-4cee-88ce-2bb79f6dbc56","Type":"ContainerDied","Data":"0323e584469a1afeda7dc01c79a16ca10b77ec662b6e45d6b27c20aa88e2bebe"} Jul 07 06:35:39.647979 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:39.647934 2579 generic.go:356] "Generic (PLEG): container finished" podID="797e47eb-12e3-4cee-88ce-2bb79f6dbc56" containerID="40d2bfe907f4c377742b19f73733b1a0b56f2f18355926323f0c5a4d38c20a7a" exitCode=0 Jul 07 06:35:39.648469 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:39.647994 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" event={"ID":"797e47eb-12e3-4cee-88ce-2bb79f6dbc56","Type":"ContainerDied","Data":"40d2bfe907f4c377742b19f73733b1a0b56f2f18355926323f0c5a4d38c20a7a"} Jul 07 06:35:40.773853 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:40.773824 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" Jul 07 06:35:40.890477 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:40.890446 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-util\") pod \"797e47eb-12e3-4cee-88ce-2bb79f6dbc56\" (UID: \"797e47eb-12e3-4cee-88ce-2bb79f6dbc56\") " Jul 07 06:35:40.890477 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:40.890481 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4crfm\" (UniqueName: \"kubernetes.io/projected/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-kube-api-access-4crfm\") pod \"797e47eb-12e3-4cee-88ce-2bb79f6dbc56\" (UID: \"797e47eb-12e3-4cee-88ce-2bb79f6dbc56\") " Jul 07 06:35:40.890676 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:40.890512 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-bundle\") pod \"797e47eb-12e3-4cee-88ce-2bb79f6dbc56\" (UID: \"797e47eb-12e3-4cee-88ce-2bb79f6dbc56\") " Jul 07 06:35:40.891389 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:40.891338 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-bundle" (OuterVolumeSpecName: "bundle") pod "797e47eb-12e3-4cee-88ce-2bb79f6dbc56" (UID: "797e47eb-12e3-4cee-88ce-2bb79f6dbc56"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:35:40.892720 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:40.892697 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-kube-api-access-4crfm" (OuterVolumeSpecName: "kube-api-access-4crfm") pod "797e47eb-12e3-4cee-88ce-2bb79f6dbc56" (UID: "797e47eb-12e3-4cee-88ce-2bb79f6dbc56"). InnerVolumeSpecName "kube-api-access-4crfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:35:40.895040 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:40.895020 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-util" (OuterVolumeSpecName: "util") pod "797e47eb-12e3-4cee-88ce-2bb79f6dbc56" (UID: "797e47eb-12e3-4cee-88ce-2bb79f6dbc56"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:35:40.991213 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:40.991147 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-bundle\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:35:40.991213 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:40.991170 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-util\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:35:40.991213 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:40.991179 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4crfm\" (UniqueName: \"kubernetes.io/projected/797e47eb-12e3-4cee-88ce-2bb79f6dbc56-kube-api-access-4crfm\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:35:41.657285 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:41.657258 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" Jul 07 06:35:41.657497 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:41.657257 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tqz8q" event={"ID":"797e47eb-12e3-4cee-88ce-2bb79f6dbc56","Type":"ContainerDied","Data":"a41955858e4ea438d130cde1798a8debe6eaa3a325b3f6104c5e9135bb4903d9"} Jul 07 06:35:41.657497 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:41.657422 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a41955858e4ea438d130cde1798a8debe6eaa3a325b3f6104c5e9135bb4903d9" Jul 07 06:35:42.782732 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:42.782700 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:43.662109 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:43.662077 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:44.806413 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:44.806382 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8mvds"] Jul 07 06:35:44.806799 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:44.806643 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="797e47eb-12e3-4cee-88ce-2bb79f6dbc56" containerName="extract" Jul 07 06:35:44.806799 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:44.806654 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="797e47eb-12e3-4cee-88ce-2bb79f6dbc56" containerName="extract" Jul 07 06:35:44.806799 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:44.806672 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="797e47eb-12e3-4cee-88ce-2bb79f6dbc56" containerName="util" Jul 07 06:35:44.806799 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:44.806677 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="797e47eb-12e3-4cee-88ce-2bb79f6dbc56" containerName="util" Jul 07 06:35:44.806799 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:44.806687 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="797e47eb-12e3-4cee-88ce-2bb79f6dbc56" containerName="pull" Jul 07 06:35:44.806799 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:44.806693 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="797e47eb-12e3-4cee-88ce-2bb79f6dbc56" containerName="pull" Jul 07 06:35:44.806799 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:44.806744 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="797e47eb-12e3-4cee-88ce-2bb79f6dbc56" containerName="extract" Jul 07 06:35:44.840014 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:44.839979 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8mvds"] Jul 07 06:35:44.840158 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:44.840083 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8mvds" Jul 07 06:35:44.842889 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:44.842868 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 06:35:44.843025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:44.842916 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 06:35:44.843339 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:44.843319 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-gjmwh\"" Jul 07 06:35:44.923603 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:44.923574 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6jbr\" (UniqueName: \"kubernetes.io/projected/d70ea693-8209-4e56-9d74-9cbf47418dd6-kube-api-access-j6jbr\") pod \"cert-manager-operator-controller-manager-55c444bf86-8mvds\" (UID: \"d70ea693-8209-4e56-9d74-9cbf47418dd6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8mvds" Jul 07 06:35:44.923783 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:44.923760 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d70ea693-8209-4e56-9d74-9cbf47418dd6-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8mvds\" (UID: \"d70ea693-8209-4e56-9d74-9cbf47418dd6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8mvds" Jul 07 06:35:45.025238 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:45.025189 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6jbr\" (UniqueName: \"kubernetes.io/projected/d70ea693-8209-4e56-9d74-9cbf47418dd6-kube-api-access-j6jbr\") pod \"cert-manager-operator-controller-manager-55c444bf86-8mvds\" (UID: \"d70ea693-8209-4e56-9d74-9cbf47418dd6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8mvds" Jul 07 06:35:45.025423 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:45.025247 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d70ea693-8209-4e56-9d74-9cbf47418dd6-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8mvds\" (UID: \"d70ea693-8209-4e56-9d74-9cbf47418dd6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8mvds" Jul 07 06:35:45.025575 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:45.025560 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d70ea693-8209-4e56-9d74-9cbf47418dd6-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8mvds\" (UID: \"d70ea693-8209-4e56-9d74-9cbf47418dd6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8mvds" Jul 07 06:35:45.033933 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:45.033901 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6jbr\" (UniqueName: \"kubernetes.io/projected/d70ea693-8209-4e56-9d74-9cbf47418dd6-kube-api-access-j6jbr\") pod \"cert-manager-operator-controller-manager-55c444bf86-8mvds\" (UID: \"d70ea693-8209-4e56-9d74-9cbf47418dd6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8mvds" Jul 07 06:35:45.148921 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:45.148841 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8mvds" Jul 07 06:35:45.299936 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:45.299857 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8mvds"] Jul 07 06:35:45.302276 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:35:45.302251 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd70ea693_8209_4e56_9d74_9cbf47418dd6.slice/crio-a0ca59a9e216351c89dc96be4c32ecdb92f324d7fabe1d6edbd7b87e20a12f75 WatchSource:0}: Error finding container a0ca59a9e216351c89dc96be4c32ecdb92f324d7fabe1d6edbd7b87e20a12f75: Status 404 returned error can't find the container with id a0ca59a9e216351c89dc96be4c32ecdb92f324d7fabe1d6edbd7b87e20a12f75 Jul 07 06:35:45.672656 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:45.672621 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8mvds" event={"ID":"d70ea693-8209-4e56-9d74-9cbf47418dd6","Type":"ContainerStarted","Data":"a0ca59a9e216351c89dc96be4c32ecdb92f324d7fabe1d6edbd7b87e20a12f75"} Jul 07 06:35:46.437615 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:46.437585 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:46.538666 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:46.538636 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:47.682691 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:47.682612 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8mvds" event={"ID":"d70ea693-8209-4e56-9d74-9cbf47418dd6","Type":"ContainerStarted","Data":"94f9840017dcf499e804b3cb454c1d3c7ef4c844c2c19a342a1854055c8dbef0"} Jul 07 06:35:47.706133 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:47.706083 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8mvds" podStartSLOduration=1.702357889 podStartE2EDuration="3.706069243s" totalImagesPullingTime="2.003711354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:35:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:35:47.704302568 +0000 UTC m=+223.544143548" watchObservedRunningTime="2026-07-07 06:35:47.706069243 +0000 UTC m=+223.545910210" Jul 07 06:35:48.189570 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:48.189536 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:48.303898 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:48.303863 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:48.831420 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:48.831390 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc"] Jul 07 06:35:48.833869 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:48.833849 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" Jul 07 06:35:48.836550 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:48.836520 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 06:35:48.837584 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:48.837557 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 06:35:48.837681 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:48.837658 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bmkmj\"" Jul 07 06:35:48.843103 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:48.843082 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc"] Jul 07 06:35:48.959025 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:48.958951 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t9wz\" (UniqueName: \"kubernetes.io/projected/f96e5310-17c8-490d-9a1a-41006b41cac5-kube-api-access-7t9wz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc\" (UID: \"f96e5310-17c8-490d-9a1a-41006b41cac5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" Jul 07 06:35:48.959158 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:48.959109 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f96e5310-17c8-490d-9a1a-41006b41cac5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc\" (UID: \"f96e5310-17c8-490d-9a1a-41006b41cac5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" Jul 07 06:35:48.959158 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:48.959144 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f96e5310-17c8-490d-9a1a-41006b41cac5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc\" (UID: \"f96e5310-17c8-490d-9a1a-41006b41cac5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" Jul 07 06:35:49.060242 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.060216 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f96e5310-17c8-490d-9a1a-41006b41cac5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc\" (UID: \"f96e5310-17c8-490d-9a1a-41006b41cac5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" Jul 07 06:35:49.060242 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.060246 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f96e5310-17c8-490d-9a1a-41006b41cac5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc\" (UID: \"f96e5310-17c8-490d-9a1a-41006b41cac5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" Jul 07 06:35:49.060449 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.060298 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t9wz\" (UniqueName: \"kubernetes.io/projected/f96e5310-17c8-490d-9a1a-41006b41cac5-kube-api-access-7t9wz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc\" (UID: \"f96e5310-17c8-490d-9a1a-41006b41cac5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" Jul 07 06:35:49.060671 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.060654 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f96e5310-17c8-490d-9a1a-41006b41cac5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc\" (UID: \"f96e5310-17c8-490d-9a1a-41006b41cac5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" Jul 07 06:35:49.060713 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.060666 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f96e5310-17c8-490d-9a1a-41006b41cac5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc\" (UID: \"f96e5310-17c8-490d-9a1a-41006b41cac5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" Jul 07 06:35:49.069065 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.069042 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t9wz\" (UniqueName: \"kubernetes.io/projected/f96e5310-17c8-490d-9a1a-41006b41cac5-kube-api-access-7t9wz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc\" (UID: \"f96e5310-17c8-490d-9a1a-41006b41cac5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" Jul 07 06:35:49.117733 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.117667 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-h9bh7"] Jul 07 06:35:49.120060 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.120043 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-h9bh7" Jul 07 06:35:49.122792 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.122763 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 06:35:49.122952 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.122932 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-pjkvf\"" Jul 07 06:35:49.123050 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.122972 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 06:35:49.130472 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.130453 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-h9bh7"] Jul 07 06:35:49.142962 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.142946 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" Jul 07 06:35:49.261773 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.261745 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4534aff6-9e1c-442e-a48d-dcb7c2d959cb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-h9bh7\" (UID: \"4534aff6-9e1c-442e-a48d-dcb7c2d959cb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h9bh7" Jul 07 06:35:49.261917 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.261812 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xczlc\" (UniqueName: \"kubernetes.io/projected/4534aff6-9e1c-442e-a48d-dcb7c2d959cb-kube-api-access-xczlc\") pod \"cert-manager-webhook-6486ffd997-h9bh7\" (UID: \"4534aff6-9e1c-442e-a48d-dcb7c2d959cb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h9bh7" Jul 07 06:35:49.271545 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.271524 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc"] Jul 07 06:35:49.273800 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:35:49.273776 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf96e5310_17c8_490d_9a1a_41006b41cac5.slice/crio-b78aab8e7e1cb16665695d6ac60bac6d96890d68252a7f115f066a2d58faf969 WatchSource:0}: Error finding container b78aab8e7e1cb16665695d6ac60bac6d96890d68252a7f115f066a2d58faf969: Status 404 returned error can't find the container with id b78aab8e7e1cb16665695d6ac60bac6d96890d68252a7f115f066a2d58faf969 Jul 07 06:35:49.362911 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.362883 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xczlc\" (UniqueName: \"kubernetes.io/projected/4534aff6-9e1c-442e-a48d-dcb7c2d959cb-kube-api-access-xczlc\") pod \"cert-manager-webhook-6486ffd997-h9bh7\" (UID: \"4534aff6-9e1c-442e-a48d-dcb7c2d959cb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h9bh7" Jul 07 06:35:49.363056 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.362929 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4534aff6-9e1c-442e-a48d-dcb7c2d959cb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-h9bh7\" (UID: \"4534aff6-9e1c-442e-a48d-dcb7c2d959cb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h9bh7" Jul 07 06:35:49.373556 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.373497 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4534aff6-9e1c-442e-a48d-dcb7c2d959cb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-h9bh7\" (UID: \"4534aff6-9e1c-442e-a48d-dcb7c2d959cb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h9bh7" Jul 07 06:35:49.373893 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.373869 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xczlc\" (UniqueName: \"kubernetes.io/projected/4534aff6-9e1c-442e-a48d-dcb7c2d959cb-kube-api-access-xczlc\") pod \"cert-manager-webhook-6486ffd997-h9bh7\" (UID: \"4534aff6-9e1c-442e-a48d-dcb7c2d959cb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h9bh7" Jul 07 06:35:49.443460 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.443436 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-h9bh7" Jul 07 06:35:49.563279 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.563255 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-h9bh7"] Jul 07 06:35:49.565537 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:35:49.565511 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4534aff6_9e1c_442e_a48d_dcb7c2d959cb.slice/crio-8f75706ae4ab72891d25308d3c238874d6c07e17cd4463d687f4b2de72e21cd2 WatchSource:0}: Error finding container 8f75706ae4ab72891d25308d3c238874d6c07e17cd4463d687f4b2de72e21cd2: Status 404 returned error can't find the container with id 8f75706ae4ab72891d25308d3c238874d6c07e17cd4463d687f4b2de72e21cd2 Jul 07 06:35:49.691495 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.691418 2579 generic.go:356] "Generic (PLEG): container finished" podID="f96e5310-17c8-490d-9a1a-41006b41cac5" containerID="87f529eaacb553bb20de22566e17a657c304ae21e35b0a5f3abd589ef6b3614c" exitCode=0 Jul 07 06:35:49.691728 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.691510 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" event={"ID":"f96e5310-17c8-490d-9a1a-41006b41cac5","Type":"ContainerDied","Data":"87f529eaacb553bb20de22566e17a657c304ae21e35b0a5f3abd589ef6b3614c"} Jul 07 06:35:49.691728 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.691571 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" event={"ID":"f96e5310-17c8-490d-9a1a-41006b41cac5","Type":"ContainerStarted","Data":"b78aab8e7e1cb16665695d6ac60bac6d96890d68252a7f115f066a2d58faf969"} Jul 07 06:35:49.692762 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:49.692737 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-h9bh7" event={"ID":"4534aff6-9e1c-442e-a48d-dcb7c2d959cb","Type":"ContainerStarted","Data":"8f75706ae4ab72891d25308d3c238874d6c07e17cd4463d687f4b2de72e21cd2"} Jul 07 06:35:50.746180 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:50.746017 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:50.952010 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:50.951970 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:51.162900 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:51.162867 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-spqz8"] Jul 07 06:35:51.165243 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:51.165218 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-spqz8" Jul 07 06:35:51.167980 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:51.167925 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-4z7ls\"" Jul 07 06:35:51.173774 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:51.173732 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-spqz8"] Jul 07 06:35:51.281310 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:51.281277 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b08a131d-d4a0-43db-bd93-035b00374531-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-spqz8\" (UID: \"b08a131d-d4a0-43db-bd93-035b00374531\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-spqz8" Jul 07 06:35:51.281502 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:51.281324 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xknpp\" (UniqueName: \"kubernetes.io/projected/b08a131d-d4a0-43db-bd93-035b00374531-kube-api-access-xknpp\") pod \"cert-manager-cainjector-559cdd8c87-spqz8\" (UID: \"b08a131d-d4a0-43db-bd93-035b00374531\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-spqz8" Jul 07 06:35:51.382703 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:51.382668 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xknpp\" (UniqueName: \"kubernetes.io/projected/b08a131d-d4a0-43db-bd93-035b00374531-kube-api-access-xknpp\") pod \"cert-manager-cainjector-559cdd8c87-spqz8\" (UID: \"b08a131d-d4a0-43db-bd93-035b00374531\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-spqz8" Jul 07 06:35:51.382863 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:51.382767 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b08a131d-d4a0-43db-bd93-035b00374531-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-spqz8\" (UID: \"b08a131d-d4a0-43db-bd93-035b00374531\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-spqz8" Jul 07 06:35:51.391944 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:51.391915 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b08a131d-d4a0-43db-bd93-035b00374531-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-spqz8\" (UID: \"b08a131d-d4a0-43db-bd93-035b00374531\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-spqz8" Jul 07 06:35:51.392266 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:51.392248 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xknpp\" (UniqueName: \"kubernetes.io/projected/b08a131d-d4a0-43db-bd93-035b00374531-kube-api-access-xknpp\") pod \"cert-manager-cainjector-559cdd8c87-spqz8\" (UID: \"b08a131d-d4a0-43db-bd93-035b00374531\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-spqz8" Jul 07 06:35:51.477896 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:51.477825 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-spqz8" Jul 07 06:35:52.882913 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:52.882886 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-spqz8"] Jul 07 06:35:52.885397 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:35:52.885346 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb08a131d_d4a0_43db_bd93_035b00374531.slice/crio-ec3cd069f9ef959680a36b967059d24acd7d7be27f634dbaa3e2a3f3aabd80e2 WatchSource:0}: Error finding container ec3cd069f9ef959680a36b967059d24acd7d7be27f634dbaa3e2a3f3aabd80e2: Status 404 returned error can't find the container with id ec3cd069f9ef959680a36b967059d24acd7d7be27f634dbaa3e2a3f3aabd80e2 Jul 07 06:35:53.388643 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:53.388612 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:53.488855 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:53.488812 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:53.713342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:53.713241 2579 generic.go:356] "Generic (PLEG): container finished" podID="f96e5310-17c8-490d-9a1a-41006b41cac5" containerID="813473f05be6a5e29acd39abe961f6839f891dea7417ae9d267ab1f7bccaee11" exitCode=0 Jul 07 06:35:53.713342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:53.713335 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" event={"ID":"f96e5310-17c8-490d-9a1a-41006b41cac5","Type":"ContainerDied","Data":"813473f05be6a5e29acd39abe961f6839f891dea7417ae9d267ab1f7bccaee11"} Jul 07 06:35:53.714944 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:53.714913 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-h9bh7" event={"ID":"4534aff6-9e1c-442e-a48d-dcb7c2d959cb","Type":"ContainerStarted","Data":"8224468ddb177d82cd400fe2ffca1ec0d35b1458a040a94a2543230687e759a4"} Jul 07 06:35:53.715077 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:53.715019 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-h9bh7" Jul 07 06:35:53.716568 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:53.716543 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-spqz8" event={"ID":"b08a131d-d4a0-43db-bd93-035b00374531","Type":"ContainerStarted","Data":"3e56646963c168f812a350cca36c2517781f0e6314a10929b33dcb5386a808e7"} Jul 07 06:35:53.716634 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:53.716583 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-spqz8" event={"ID":"b08a131d-d4a0-43db-bd93-035b00374531","Type":"ContainerStarted","Data":"ec3cd069f9ef959680a36b967059d24acd7d7be27f634dbaa3e2a3f3aabd80e2"} Jul 07 06:35:53.751426 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:53.751348 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-h9bh7" podStartSLOduration=1.559669987 podStartE2EDuration="4.751334982s" totalImagesPullingTime="3.191664995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:35:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:35:53.750816123 +0000 UTC m=+229.590657091" watchObservedRunningTime="2026-07-07 06:35:53.751334982 +0000 UTC m=+229.591175949" Jul 07 06:35:53.776311 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:53.776259 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-spqz8" podStartSLOduration=2.776245123 podStartE2EDuration="2.776245123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:35:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:35:53.774354544 +0000 UTC m=+229.614195527" watchObservedRunningTime="2026-07-07 06:35:53.776245123 +0000 UTC m=+229.616086089" Jul 07 06:35:54.727488 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:54.727452 2579 generic.go:356] "Generic (PLEG): container finished" podID="f96e5310-17c8-490d-9a1a-41006b41cac5" containerID="dc5431150bdf84514a2f2182cc92b0e858467df7e346a68115f6966e1a88ec37" exitCode=0 Jul 07 06:35:54.727930 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:54.727525 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" event={"ID":"f96e5310-17c8-490d-9a1a-41006b41cac5","Type":"ContainerDied","Data":"dc5431150bdf84514a2f2182cc92b0e858467df7e346a68115f6966e1a88ec37"} Jul 07 06:35:55.850306 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:55.850283 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" Jul 07 06:35:55.918600 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:55.918572 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f96e5310-17c8-490d-9a1a-41006b41cac5-bundle\") pod \"f96e5310-17c8-490d-9a1a-41006b41cac5\" (UID: \"f96e5310-17c8-490d-9a1a-41006b41cac5\") " Jul 07 06:35:55.918768 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:55.918639 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f96e5310-17c8-490d-9a1a-41006b41cac5-util\") pod \"f96e5310-17c8-490d-9a1a-41006b41cac5\" (UID: \"f96e5310-17c8-490d-9a1a-41006b41cac5\") " Jul 07 06:35:55.918768 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:55.918662 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7t9wz\" (UniqueName: \"kubernetes.io/projected/f96e5310-17c8-490d-9a1a-41006b41cac5-kube-api-access-7t9wz\") pod \"f96e5310-17c8-490d-9a1a-41006b41cac5\" (UID: \"f96e5310-17c8-490d-9a1a-41006b41cac5\") " Jul 07 06:35:55.919009 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:55.918985 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f96e5310-17c8-490d-9a1a-41006b41cac5-bundle" (OuterVolumeSpecName: "bundle") pod "f96e5310-17c8-490d-9a1a-41006b41cac5" (UID: "f96e5310-17c8-490d-9a1a-41006b41cac5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:35:55.920667 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:55.920646 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f96e5310-17c8-490d-9a1a-41006b41cac5-kube-api-access-7t9wz" (OuterVolumeSpecName: "kube-api-access-7t9wz") pod "f96e5310-17c8-490d-9a1a-41006b41cac5" (UID: "f96e5310-17c8-490d-9a1a-41006b41cac5"). InnerVolumeSpecName "kube-api-access-7t9wz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:35:55.922673 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:55.922654 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f96e5310-17c8-490d-9a1a-41006b41cac5-util" (OuterVolumeSpecName: "util") pod "f96e5310-17c8-490d-9a1a-41006b41cac5" (UID: "f96e5310-17c8-490d-9a1a-41006b41cac5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:35:56.020146 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:56.020079 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f96e5310-17c8-490d-9a1a-41006b41cac5-util\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:35:56.020146 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:56.020102 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t9wz\" (UniqueName: \"kubernetes.io/projected/f96e5310-17c8-490d-9a1a-41006b41cac5-kube-api-access-7t9wz\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:35:56.020146 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:56.020112 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f96e5310-17c8-490d-9a1a-41006b41cac5-bundle\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:35:56.737586 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:56.737551 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" event={"ID":"f96e5310-17c8-490d-9a1a-41006b41cac5","Type":"ContainerDied","Data":"b78aab8e7e1cb16665695d6ac60bac6d96890d68252a7f115f066a2d58faf969"} Jul 07 06:35:56.737586 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:56.737590 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb4hcc" Jul 07 06:35:56.737783 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:56.737600 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b78aab8e7e1cb16665695d6ac60bac6d96890d68252a7f115f066a2d58faf969" Jul 07 06:35:57.957063 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:57.957033 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:35:59.729904 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:35:59.729874 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-h9bh7" Jul 07 06:36:04.813926 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:04.813893 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:04.906376 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:04.906340 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:07.031222 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.031185 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lpmmz"] Jul 07 06:36:07.031709 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.031509 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f96e5310-17c8-490d-9a1a-41006b41cac5" containerName="util" Jul 07 06:36:07.031709 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.031525 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f96e5310-17c8-490d-9a1a-41006b41cac5" containerName="util" Jul 07 06:36:07.031709 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.031549 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f96e5310-17c8-490d-9a1a-41006b41cac5" containerName="extract" Jul 07 06:36:07.031709 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.031554 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f96e5310-17c8-490d-9a1a-41006b41cac5" containerName="extract" Jul 07 06:36:07.031709 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.031562 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f96e5310-17c8-490d-9a1a-41006b41cac5" containerName="pull" Jul 07 06:36:07.031709 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.031567 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f96e5310-17c8-490d-9a1a-41006b41cac5" containerName="pull" Jul 07 06:36:07.031709 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.031614 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f96e5310-17c8-490d-9a1a-41006b41cac5" containerName="extract" Jul 07 06:36:07.034780 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.034762 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lpmmz" Jul 07 06:36:07.037681 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.037661 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 06:36:07.038754 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.038726 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-r6gxw\"" Jul 07 06:36:07.038869 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.038756 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 06:36:07.045132 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.045084 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lpmmz"] Jul 07 06:36:07.105226 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.105193 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0157843-d343-406e-ba0a-c10de7dd32cb-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lpmmz\" (UID: \"f0157843-d343-406e-ba0a-c10de7dd32cb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lpmmz" Jul 07 06:36:07.105385 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.105250 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjt6v\" (UniqueName: \"kubernetes.io/projected/f0157843-d343-406e-ba0a-c10de7dd32cb-kube-api-access-xjt6v\") pod \"openshift-lws-operator-fd8ccff4c-lpmmz\" (UID: \"f0157843-d343-406e-ba0a-c10de7dd32cb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lpmmz" Jul 07 06:36:07.116896 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.116865 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:07.205875 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.205840 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjt6v\" (UniqueName: \"kubernetes.io/projected/f0157843-d343-406e-ba0a-c10de7dd32cb-kube-api-access-xjt6v\") pod \"openshift-lws-operator-fd8ccff4c-lpmmz\" (UID: \"f0157843-d343-406e-ba0a-c10de7dd32cb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lpmmz" Jul 07 06:36:07.206047 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.205919 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0157843-d343-406e-ba0a-c10de7dd32cb-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lpmmz\" (UID: \"f0157843-d343-406e-ba0a-c10de7dd32cb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lpmmz" Jul 07 06:36:07.206313 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.206293 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0157843-d343-406e-ba0a-c10de7dd32cb-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lpmmz\" (UID: \"f0157843-d343-406e-ba0a-c10de7dd32cb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lpmmz" Jul 07 06:36:07.214548 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.214527 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjt6v\" (UniqueName: \"kubernetes.io/projected/f0157843-d343-406e-ba0a-c10de7dd32cb-kube-api-access-xjt6v\") pod \"openshift-lws-operator-fd8ccff4c-lpmmz\" (UID: \"f0157843-d343-406e-ba0a-c10de7dd32cb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lpmmz" Jul 07 06:36:07.215023 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.214988 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:07.346004 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.345904 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lpmmz" Jul 07 06:36:07.475899 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.475837 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lpmmz"] Jul 07 06:36:07.478077 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:36:07.478051 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0157843_d343_406e_ba0a_c10de7dd32cb.slice/crio-3ae14db4966a35b69911cbb071cabf988f03fc52af6dfd42e6cf475bb8583f72 WatchSource:0}: Error finding container 3ae14db4966a35b69911cbb071cabf988f03fc52af6dfd42e6cf475bb8583f72: Status 404 returned error can't find the container with id 3ae14db4966a35b69911cbb071cabf988f03fc52af6dfd42e6cf475bb8583f72 Jul 07 06:36:07.790352 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:07.790317 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lpmmz" event={"ID":"f0157843-d343-406e-ba0a-c10de7dd32cb","Type":"ContainerStarted","Data":"3ae14db4966a35b69911cbb071cabf988f03fc52af6dfd42e6cf475bb8583f72"} Jul 07 06:36:09.801156 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:09.801121 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lpmmz" event={"ID":"f0157843-d343-406e-ba0a-c10de7dd32cb","Type":"ContainerStarted","Data":"e4d23fbcc6b7df9f2ea72511988b242d7e979c9231241cf6ae8bf4e9e91d9fc5"} Jul 07 06:36:09.818231 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:09.818174 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lpmmz" podStartSLOduration=0.900416465 podStartE2EDuration="2.818154414s" totalImagesPullingTime="1.917737949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:36:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:36:09.818061722 +0000 UTC m=+245.657902690" watchObservedRunningTime="2026-07-07 06:36:09.818154414 +0000 UTC m=+245.657995382" Jul 07 06:36:10.533248 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:10.533215 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:10.632528 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:10.632495 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:12.552112 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:12.552073 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:14.279534 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.279497 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv"] Jul 07 06:36:14.281785 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.281768 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" Jul 07 06:36:14.284520 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.284497 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 06:36:14.284520 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.284512 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 06:36:14.285635 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.285614 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bmkmj\"" Jul 07 06:36:14.290352 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.290329 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv"] Jul 07 06:36:14.372408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.372340 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv\" (UID: \"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" Jul 07 06:36:14.372589 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.372424 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fmg7\" (UniqueName: \"kubernetes.io/projected/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-kube-api-access-7fmg7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv\" (UID: \"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" Jul 07 06:36:14.372589 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.372500 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv\" (UID: \"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" Jul 07 06:36:14.473472 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.473438 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fmg7\" (UniqueName: \"kubernetes.io/projected/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-kube-api-access-7fmg7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv\" (UID: \"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" Jul 07 06:36:14.473616 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.473505 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv\" (UID: \"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" Jul 07 06:36:14.473616 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.473565 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv\" (UID: \"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" Jul 07 06:36:14.473830 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.473812 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv\" (UID: \"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" Jul 07 06:36:14.473886 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.473869 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv\" (UID: \"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" Jul 07 06:36:14.481870 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.481847 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fmg7\" (UniqueName: \"kubernetes.io/projected/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-kube-api-access-7fmg7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv\" (UID: \"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" Jul 07 06:36:14.591629 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.591534 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" Jul 07 06:36:14.718480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.718459 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv"] Jul 07 06:36:14.720150 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:36:14.720121 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2069e4d_e2a9_4d36_901b_4ab2ee0beedc.slice/crio-4714d6c4bea2cfd2ad7d363b1553a9226b3d50b4c3866eebece3fd6fe5c93ce1 WatchSource:0}: Error finding container 4714d6c4bea2cfd2ad7d363b1553a9226b3d50b4c3866eebece3fd6fe5c93ce1: Status 404 returned error can't find the container with id 4714d6c4bea2cfd2ad7d363b1553a9226b3d50b4c3866eebece3fd6fe5c93ce1 Jul 07 06:36:14.829657 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.829615 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" event={"ID":"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc","Type":"ContainerStarted","Data":"56c809fe27e3e8fa80c09522fd4f666aa6a96f2b1de15591b1420a9fce71df59"} Jul 07 06:36:14.829854 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:14.829689 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" event={"ID":"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc","Type":"ContainerStarted","Data":"4714d6c4bea2cfd2ad7d363b1553a9226b3d50b4c3866eebece3fd6fe5c93ce1"} Jul 07 06:36:15.834866 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:15.834835 2579 generic.go:356] "Generic (PLEG): container finished" podID="c2069e4d-e2a9-4d36-901b-4ab2ee0beedc" containerID="56c809fe27e3e8fa80c09522fd4f666aa6a96f2b1de15591b1420a9fce71df59" exitCode=0 Jul 07 06:36:15.835299 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:15.834922 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" event={"ID":"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc","Type":"ContainerDied","Data":"56c809fe27e3e8fa80c09522fd4f666aa6a96f2b1de15591b1420a9fce71df59"} Jul 07 06:36:16.840612 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:16.840575 2579 generic.go:356] "Generic (PLEG): container finished" podID="c2069e4d-e2a9-4d36-901b-4ab2ee0beedc" containerID="b517007d321a15571f8c9762f208e14580237f835b86620a12655f083126677f" exitCode=0 Jul 07 06:36:16.840975 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:16.840631 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" event={"ID":"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc","Type":"ContainerDied","Data":"b517007d321a15571f8c9762f208e14580237f835b86620a12655f083126677f"} Jul 07 06:36:17.846880 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:17.846848 2579 generic.go:356] "Generic (PLEG): container finished" podID="c2069e4d-e2a9-4d36-901b-4ab2ee0beedc" containerID="f02b3c2f91d9fec77936dfe775d5f37969049735925bbd605abc50adcc9a43ef" exitCode=0 Jul 07 06:36:17.847275 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:17.846929 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" event={"ID":"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc","Type":"ContainerDied","Data":"f02b3c2f91d9fec77936dfe775d5f37969049735925bbd605abc50adcc9a43ef"} Jul 07 06:36:18.975620 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:18.975597 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" Jul 07 06:36:19.110842 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:19.110762 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fmg7\" (UniqueName: \"kubernetes.io/projected/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-kube-api-access-7fmg7\") pod \"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc\" (UID: \"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc\") " Jul 07 06:36:19.110842 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:19.110799 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-bundle\") pod \"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc\" (UID: \"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc\") " Jul 07 06:36:19.111027 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:19.110925 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-util\") pod \"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc\" (UID: \"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc\") " Jul 07 06:36:19.111661 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:19.111636 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-bundle" (OuterVolumeSpecName: "bundle") pod "c2069e4d-e2a9-4d36-901b-4ab2ee0beedc" (UID: "c2069e4d-e2a9-4d36-901b-4ab2ee0beedc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:36:19.113140 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:19.113115 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-kube-api-access-7fmg7" (OuterVolumeSpecName: "kube-api-access-7fmg7") pod "c2069e4d-e2a9-4d36-901b-4ab2ee0beedc" (UID: "c2069e4d-e2a9-4d36-901b-4ab2ee0beedc"). InnerVolumeSpecName "kube-api-access-7fmg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:36:19.116576 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:19.116548 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-util" (OuterVolumeSpecName: "util") pod "c2069e4d-e2a9-4d36-901b-4ab2ee0beedc" (UID: "c2069e4d-e2a9-4d36-901b-4ab2ee0beedc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:36:19.211649 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:19.211616 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-util\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:36:19.211649 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:19.211645 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fmg7\" (UniqueName: \"kubernetes.io/projected/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-kube-api-access-7fmg7\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:36:19.211649 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:19.211656 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2069e4d-e2a9-4d36-901b-4ab2ee0beedc-bundle\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:36:19.293657 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:19.293623 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:19.858010 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:19.857974 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" event={"ID":"c2069e4d-e2a9-4d36-901b-4ab2ee0beedc","Type":"ContainerDied","Data":"4714d6c4bea2cfd2ad7d363b1553a9226b3d50b4c3866eebece3fd6fe5c93ce1"} Jul 07 06:36:19.858210 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:19.858027 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4714d6c4bea2cfd2ad7d363b1553a9226b3d50b4c3866eebece3fd6fe5c93ce1" Jul 07 06:36:19.858210 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:19.857987 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmk2dv" Jul 07 06:36:24.758645 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.758608 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:24.820745 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.820714 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr"] Jul 07 06:36:24.820998 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.820987 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2069e4d-e2a9-4d36-901b-4ab2ee0beedc" containerName="extract" Jul 07 06:36:24.821040 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.820999 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2069e4d-e2a9-4d36-901b-4ab2ee0beedc" containerName="extract" Jul 07 06:36:24.821040 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.821009 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2069e4d-e2a9-4d36-901b-4ab2ee0beedc" containerName="pull" Jul 07 06:36:24.821040 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.821014 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2069e4d-e2a9-4d36-901b-4ab2ee0beedc" containerName="pull" Jul 07 06:36:24.821040 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.821029 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2069e4d-e2a9-4d36-901b-4ab2ee0beedc" containerName="util" Jul 07 06:36:24.821040 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.821035 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2069e4d-e2a9-4d36-901b-4ab2ee0beedc" containerName="util" Jul 07 06:36:24.821186 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.821097 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2069e4d-e2a9-4d36-901b-4ab2ee0beedc" containerName="extract" Jul 07 06:36:24.824783 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.824763 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:24.828999 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.828974 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 06:36:24.829138 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.828974 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lc28m\"" Jul 07 06:36:24.829214 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.829137 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 06:36:24.829214 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.829161 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 06:36:24.835292 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.835270 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr"] Jul 07 06:36:24.845738 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.845717 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:24.960457 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.960412 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a32693bd-898c-41e1-b8da-83caea138314-metrics-cert\") pod \"lws-controller-manager-856bfb9fcf-8csnr\" (UID: \"a32693bd-898c-41e1-b8da-83caea138314\") " pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:24.960617 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.960532 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a32693bd-898c-41e1-b8da-83caea138314-manager-config\") pod \"lws-controller-manager-856bfb9fcf-8csnr\" (UID: \"a32693bd-898c-41e1-b8da-83caea138314\") " pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:24.960617 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.960609 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a32693bd-898c-41e1-b8da-83caea138314-cert\") pod \"lws-controller-manager-856bfb9fcf-8csnr\" (UID: \"a32693bd-898c-41e1-b8da-83caea138314\") " pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:24.960691 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:24.960635 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cwbq\" (UniqueName: \"kubernetes.io/projected/a32693bd-898c-41e1-b8da-83caea138314-kube-api-access-6cwbq\") pod \"lws-controller-manager-856bfb9fcf-8csnr\" (UID: \"a32693bd-898c-41e1-b8da-83caea138314\") " pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:25.061176 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:25.061143 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a32693bd-898c-41e1-b8da-83caea138314-manager-config\") pod \"lws-controller-manager-856bfb9fcf-8csnr\" (UID: \"a32693bd-898c-41e1-b8da-83caea138314\") " pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:25.061358 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:25.061218 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a32693bd-898c-41e1-b8da-83caea138314-cert\") pod \"lws-controller-manager-856bfb9fcf-8csnr\" (UID: \"a32693bd-898c-41e1-b8da-83caea138314\") " pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:25.061358 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:25.061248 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cwbq\" (UniqueName: \"kubernetes.io/projected/a32693bd-898c-41e1-b8da-83caea138314-kube-api-access-6cwbq\") pod \"lws-controller-manager-856bfb9fcf-8csnr\" (UID: \"a32693bd-898c-41e1-b8da-83caea138314\") " pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:25.061358 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:25.061288 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a32693bd-898c-41e1-b8da-83caea138314-metrics-cert\") pod \"lws-controller-manager-856bfb9fcf-8csnr\" (UID: \"a32693bd-898c-41e1-b8da-83caea138314\") " pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:25.061928 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:25.061890 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a32693bd-898c-41e1-b8da-83caea138314-manager-config\") pod \"lws-controller-manager-856bfb9fcf-8csnr\" (UID: \"a32693bd-898c-41e1-b8da-83caea138314\") " pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:25.064030 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:25.064003 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a32693bd-898c-41e1-b8da-83caea138314-cert\") pod \"lws-controller-manager-856bfb9fcf-8csnr\" (UID: \"a32693bd-898c-41e1-b8da-83caea138314\") " pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:25.068875 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:25.068846 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a32693bd-898c-41e1-b8da-83caea138314-metrics-cert\") pod \"lws-controller-manager-856bfb9fcf-8csnr\" (UID: \"a32693bd-898c-41e1-b8da-83caea138314\") " pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:25.071691 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:25.071662 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cwbq\" (UniqueName: \"kubernetes.io/projected/a32693bd-898c-41e1-b8da-83caea138314-kube-api-access-6cwbq\") pod \"lws-controller-manager-856bfb9fcf-8csnr\" (UID: \"a32693bd-898c-41e1-b8da-83caea138314\") " pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:25.133621 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:25.133596 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:25.280530 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:25.280509 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr"] Jul 07 06:36:25.282545 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:36:25.282519 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda32693bd_898c_41e1_b8da_83caea138314.slice/crio-bc33dec4ea6aa36dbd771a89e0fbc2e822391e16e5e52231c7543bf559aeecc2 WatchSource:0}: Error finding container bc33dec4ea6aa36dbd771a89e0fbc2e822391e16e5e52231c7543bf559aeecc2: Status 404 returned error can't find the container with id bc33dec4ea6aa36dbd771a89e0fbc2e822391e16e5e52231c7543bf559aeecc2 Jul 07 06:36:25.883304 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:25.883268 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" event={"ID":"a32693bd-898c-41e1-b8da-83caea138314","Type":"ContainerStarted","Data":"bc33dec4ea6aa36dbd771a89e0fbc2e822391e16e5e52231c7543bf559aeecc2"} Jul 07 06:36:26.889509 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:26.889467 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" event={"ID":"a32693bd-898c-41e1-b8da-83caea138314","Type":"ContainerStarted","Data":"c584da6f67da28d196d2d253b62fe40834cdb4d2ba425219a0b33ed9650637d7"} Jul 07 06:36:26.889873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:26.889539 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:26.918877 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:26.918826 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" podStartSLOduration=1.4254379830000001 podStartE2EDuration="2.918810053s" totalImagesPullingTime="1.49337207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:36:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:36:26.91811088 +0000 UTC m=+262.757951847" watchObservedRunningTime="2026-07-07 06:36:26.918810053 +0000 UTC m=+262.758651045" Jul 07 06:36:29.606414 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:29.606373 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:30.596751 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.596716 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw"] Jul 07 06:36:30.599547 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.599528 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.604481 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.604458 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 06:36:30.604802 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.604716 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 06:36:30.604802 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.604735 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 06:36:30.604953 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.604851 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mhfqd\"" Jul 07 06:36:30.605271 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.605255 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 06:36:30.619997 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.619975 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw"] Jul 07 06:36:30.713574 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.713543 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1cd7caa7-5d44-4786-bda1-ef41b106db29-tmp\") pod \"opendatahub-operator-controller-manager-c66c8b568-ljkmw\" (UID: \"1cd7caa7-5d44-4786-bda1-ef41b106db29\") " pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.713760 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.713587 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1cd7caa7-5d44-4786-bda1-ef41b106db29-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c66c8b568-ljkmw\" (UID: \"1cd7caa7-5d44-4786-bda1-ef41b106db29\") " pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.713760 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.713645 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cd7caa7-5d44-4786-bda1-ef41b106db29-webhook-cert\") pod \"opendatahub-operator-controller-manager-c66c8b568-ljkmw\" (UID: \"1cd7caa7-5d44-4786-bda1-ef41b106db29\") " pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.713760 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.713707 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1cd7caa7-5d44-4786-bda1-ef41b106db29-manifests\") pod \"opendatahub-operator-controller-manager-c66c8b568-ljkmw\" (UID: \"1cd7caa7-5d44-4786-bda1-ef41b106db29\") " pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.713760 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.713741 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw2cg\" (UniqueName: \"kubernetes.io/projected/1cd7caa7-5d44-4786-bda1-ef41b106db29-kube-api-access-qw2cg\") pod \"opendatahub-operator-controller-manager-c66c8b568-ljkmw\" (UID: \"1cd7caa7-5d44-4786-bda1-ef41b106db29\") " pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.814466 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.814428 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw2cg\" (UniqueName: \"kubernetes.io/projected/1cd7caa7-5d44-4786-bda1-ef41b106db29-kube-api-access-qw2cg\") pod \"opendatahub-operator-controller-manager-c66c8b568-ljkmw\" (UID: \"1cd7caa7-5d44-4786-bda1-ef41b106db29\") " pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.814652 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.814484 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1cd7caa7-5d44-4786-bda1-ef41b106db29-tmp\") pod \"opendatahub-operator-controller-manager-c66c8b568-ljkmw\" (UID: \"1cd7caa7-5d44-4786-bda1-ef41b106db29\") " pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.814652 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.814519 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1cd7caa7-5d44-4786-bda1-ef41b106db29-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c66c8b568-ljkmw\" (UID: \"1cd7caa7-5d44-4786-bda1-ef41b106db29\") " pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.814652 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.814550 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cd7caa7-5d44-4786-bda1-ef41b106db29-webhook-cert\") pod \"opendatahub-operator-controller-manager-c66c8b568-ljkmw\" (UID: \"1cd7caa7-5d44-4786-bda1-ef41b106db29\") " pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.814652 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.814585 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1cd7caa7-5d44-4786-bda1-ef41b106db29-manifests\") pod \"opendatahub-operator-controller-manager-c66c8b568-ljkmw\" (UID: \"1cd7caa7-5d44-4786-bda1-ef41b106db29\") " pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.814973 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.814949 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1cd7caa7-5d44-4786-bda1-ef41b106db29-manifests\") pod \"opendatahub-operator-controller-manager-c66c8b568-ljkmw\" (UID: \"1cd7caa7-5d44-4786-bda1-ef41b106db29\") " pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.816712 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.816681 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1cd7caa7-5d44-4786-bda1-ef41b106db29-tmp\") pod \"opendatahub-operator-controller-manager-c66c8b568-ljkmw\" (UID: \"1cd7caa7-5d44-4786-bda1-ef41b106db29\") " pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.816964 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.816945 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1cd7caa7-5d44-4786-bda1-ef41b106db29-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c66c8b568-ljkmw\" (UID: \"1cd7caa7-5d44-4786-bda1-ef41b106db29\") " pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.817038 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.816944 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cd7caa7-5d44-4786-bda1-ef41b106db29-webhook-cert\") pod \"opendatahub-operator-controller-manager-c66c8b568-ljkmw\" (UID: \"1cd7caa7-5d44-4786-bda1-ef41b106db29\") " pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.826040 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.826009 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw2cg\" (UniqueName: \"kubernetes.io/projected/1cd7caa7-5d44-4786-bda1-ef41b106db29-kube-api-access-qw2cg\") pod \"opendatahub-operator-controller-manager-c66c8b568-ljkmw\" (UID: \"1cd7caa7-5d44-4786-bda1-ef41b106db29\") " pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:30.910090 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:30.910015 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:31.064424 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:31.064393 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw"] Jul 07 06:36:31.067826 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:36:31.067796 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd7caa7_5d44_4786_bda1_ef41b106db29.slice/crio-3027a20c57ce4fdb15499a42565111785669bbc9fc77720d68b5ca58bafb89d9 WatchSource:0}: Error finding container 3027a20c57ce4fdb15499a42565111785669bbc9fc77720d68b5ca58bafb89d9: Status 404 returned error can't find the container with id 3027a20c57ce4fdb15499a42565111785669bbc9fc77720d68b5ca58bafb89d9 Jul 07 06:36:31.281863 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:31.281825 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:31.913197 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:31.913161 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" event={"ID":"1cd7caa7-5d44-4786-bda1-ef41b106db29","Type":"ContainerStarted","Data":"3027a20c57ce4fdb15499a42565111785669bbc9fc77720d68b5ca58bafb89d9"} Jul 07 06:36:34.937614 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:34.937565 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" event={"ID":"1cd7caa7-5d44-4786-bda1-ef41b106db29","Type":"ContainerStarted","Data":"bcceb409b90083abfdb06f4339c05b768775f80236761560521d6c676b823ef2"} Jul 07 06:36:35.943322 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:35.943290 2579 generic.go:356] "Generic (PLEG): container finished" podID="1cd7caa7-5d44-4786-bda1-ef41b106db29" containerID="bcceb409b90083abfdb06f4339c05b768775f80236761560521d6c676b823ef2" exitCode=0 Jul 07 06:36:35.943710 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:35.943331 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" event={"ID":"1cd7caa7-5d44-4786-bda1-ef41b106db29","Type":"ContainerDied","Data":"bcceb409b90083abfdb06f4339c05b768775f80236761560521d6c676b823ef2"} Jul 07 06:36:36.949767 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:36.949730 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" event={"ID":"1cd7caa7-5d44-4786-bda1-ef41b106db29","Type":"ContainerStarted","Data":"fd7bc74b531177c08c7d53bb3d5c5cd6558fb4324a1df57b051317834adbd846"} Jul 07 06:36:36.950197 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:36.949881 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:36.974764 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:36.974711 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" podStartSLOduration=2.578048142 podStartE2EDuration="6.974695114s" totalImagesPullingTime="3.367582717s" totalInitContainerRuntime="1.029064255s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:36:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:36:36.973663861 +0000 UTC m=+272.813504829" watchObservedRunningTime="2026-07-07 06:36:36.974695114 +0000 UTC m=+272.814536144" Jul 07 06:36:37.896033 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:37.896006 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-856bfb9fcf-8csnr" Jul 07 06:36:42.420875 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:42.420829 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:42.519061 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:42.519011 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:47.956793 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:47.956764 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c66c8b568-ljkmw" Jul 07 06:36:49.940696 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:49.940666 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:50.026414 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:50.026357 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:50.219687 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:50.219611 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:50.651250 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:50.651209 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:50.937916 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:50.937831 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:51.051534 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:51.051497 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:51.196861 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:51.196779 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:51.485360 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:51.485279 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:52.270588 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:52.270554 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:52.388072 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:52.388040 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:52.492075 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:52.492042 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:53.414713 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:53.414668 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:57.707862 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:57.707825 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:57.811187 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:57.811153 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:58.427096 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.427061 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5"] Jul 07 06:36:58.430445 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.430429 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.433332 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.433312 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-6s826\"" Jul 07 06:36:58.433996 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.433980 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 06:36:58.434063 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.434029 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 06:36:58.434308 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.434291 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 06:36:58.434487 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.434467 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1057d10c-fc71-480b-8f6a-1a6f4832748c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.434555 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.434525 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1057d10c-fc71-480b-8f6a-1a6f4832748c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.434611 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.434564 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1057d10c-fc71-480b-8f6a-1a6f4832748c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.434611 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.434599 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1057d10c-fc71-480b-8f6a-1a6f4832748c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.434685 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.434655 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057d10c-fc71-480b-8f6a-1a6f4832748c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.434719 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.434684 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057d10c-fc71-480b-8f6a-1a6f4832748c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.434719 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.434701 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96h2v\" (UniqueName: \"kubernetes.io/projected/1057d10c-fc71-480b-8f6a-1a6f4832748c-kube-api-access-96h2v\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.434796 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.434734 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1057d10c-fc71-480b-8f6a-1a6f4832748c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.434796 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.434750 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1057d10c-fc71-480b-8f6a-1a6f4832748c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.444943 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.444922 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5"] Jul 07 06:36:58.471488 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.471460 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:58.535522 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.535494 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057d10c-fc71-480b-8f6a-1a6f4832748c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.535638 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.535532 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057d10c-fc71-480b-8f6a-1a6f4832748c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.535638 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.535550 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96h2v\" (UniqueName: \"kubernetes.io/projected/1057d10c-fc71-480b-8f6a-1a6f4832748c-kube-api-access-96h2v\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.535638 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.535593 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1057d10c-fc71-480b-8f6a-1a6f4832748c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.535638 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.535617 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1057d10c-fc71-480b-8f6a-1a6f4832748c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.535844 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.535663 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1057d10c-fc71-480b-8f6a-1a6f4832748c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.535844 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.535688 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1057d10c-fc71-480b-8f6a-1a6f4832748c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.535844 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.535731 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1057d10c-fc71-480b-8f6a-1a6f4832748c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.535844 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.535772 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1057d10c-fc71-480b-8f6a-1a6f4832748c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.536023 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.535947 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057d10c-fc71-480b-8f6a-1a6f4832748c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.536023 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.535965 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1057d10c-fc71-480b-8f6a-1a6f4832748c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.536149 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.536121 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1057d10c-fc71-480b-8f6a-1a6f4832748c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.536315 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.536296 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1057d10c-fc71-480b-8f6a-1a6f4832748c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.536407 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.536393 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1057d10c-fc71-480b-8f6a-1a6f4832748c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.537895 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.537876 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1057d10c-fc71-480b-8f6a-1a6f4832748c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.538221 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.538206 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1057d10c-fc71-480b-8f6a-1a6f4832748c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.544170 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.544149 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1057d10c-fc71-480b-8f6a-1a6f4832748c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.544261 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.544194 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96h2v\" (UniqueName: \"kubernetes.io/projected/1057d10c-fc71-480b-8f6a-1a6f4832748c-kube-api-access-96h2v\") pod \"data-science-gateway-data-science-gateway-class-6576859cf42bvt5\" (UID: \"1057d10c-fc71-480b-8f6a-1a6f4832748c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.633905 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.633866 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:58.729817 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.729739 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:58.740958 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.740931 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:36:58.839857 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.839828 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:58.882931 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.882901 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5"] Jul 07 06:36:58.886635 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:36:58.886601 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1057d10c_fc71_480b_8f6a_1a6f4832748c.slice/crio-0d3fb6f5a088a37ea673543783a3aa70ae4af3a87a39770e96354de8f8ab7170 WatchSource:0}: Error finding container 0d3fb6f5a088a37ea673543783a3aa70ae4af3a87a39770e96354de8f8ab7170: Status 404 returned error can't find the container with id 0d3fb6f5a088a37ea673543783a3aa70ae4af3a87a39770e96354de8f8ab7170 Jul 07 06:36:58.957573 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:58.957541 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:36:59.056206 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:36:59.056172 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" event={"ID":"1057d10c-fc71-480b-8f6a-1a6f4832748c","Type":"ContainerStarted","Data":"0d3fb6f5a088a37ea673543783a3aa70ae4af3a87a39770e96354de8f8ab7170"} Jul 07 06:37:00.434258 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.434224 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq"] Jul 07 06:37:00.437851 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.437832 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.440908 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.440882 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 06:37:00.441039 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.440882 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 06:37:00.441039 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.440892 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 06:37:00.441156 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.441085 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cljnv\"" Jul 07 06:37:00.452219 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.452189 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/230065e6-ce67-4fbc-95bf-dff89318b04a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.452338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.452243 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/230065e6-ce67-4fbc-95bf-dff89318b04a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.452338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.452297 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/230065e6-ce67-4fbc-95bf-dff89318b04a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.452509 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.452338 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrf86\" (UniqueName: \"kubernetes.io/projected/230065e6-ce67-4fbc-95bf-dff89318b04a-kube-api-access-mrf86\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.452509 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.452459 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/230065e6-ce67-4fbc-95bf-dff89318b04a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.452509 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.452488 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/230065e6-ce67-4fbc-95bf-dff89318b04a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.452677 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.452523 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/230065e6-ce67-4fbc-95bf-dff89318b04a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.453410 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.453389 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq"] Jul 07 06:37:00.553295 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.553259 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/230065e6-ce67-4fbc-95bf-dff89318b04a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.553510 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.553304 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/230065e6-ce67-4fbc-95bf-dff89318b04a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.553510 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.553336 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/230065e6-ce67-4fbc-95bf-dff89318b04a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.553510 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.553398 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/230065e6-ce67-4fbc-95bf-dff89318b04a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.553510 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.553432 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/230065e6-ce67-4fbc-95bf-dff89318b04a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.553510 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.553465 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/230065e6-ce67-4fbc-95bf-dff89318b04a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.553510 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.553504 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrf86\" (UniqueName: \"kubernetes.io/projected/230065e6-ce67-4fbc-95bf-dff89318b04a-kube-api-access-mrf86\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.554535 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.554492 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/230065e6-ce67-4fbc-95bf-dff89318b04a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.556327 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.556301 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/230065e6-ce67-4fbc-95bf-dff89318b04a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.556496 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.556417 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/230065e6-ce67-4fbc-95bf-dff89318b04a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.556496 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.556455 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/230065e6-ce67-4fbc-95bf-dff89318b04a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.556685 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.556663 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/230065e6-ce67-4fbc-95bf-dff89318b04a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.564379 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.564341 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/230065e6-ce67-4fbc-95bf-dff89318b04a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.564940 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.564918 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrf86\" (UniqueName: \"kubernetes.io/projected/230065e6-ce67-4fbc-95bf-dff89318b04a-kube-api-access-mrf86\") pod \"istiod-openshift-gateway-94bb8fbfd-2gjqq\" (UID: \"230065e6-ce67-4fbc-95bf-dff89318b04a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:00.749873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:00.749798 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:01.522879 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:01.522844 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:37:01.523166 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:01.522918 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:37:01.523166 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:01.522952 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:37:01.597737 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:01.597710 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq"] Jul 07 06:37:01.599960 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:37:01.599922 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod230065e6_ce67_4fbc_95bf_dff89318b04a.slice/crio-1b52d4523b0f70b5ea3ecd2c5c8a630605f719a86dffde6a6065bc4c903feb6c WatchSource:0}: Error finding container 1b52d4523b0f70b5ea3ecd2c5c8a630605f719a86dffde6a6065bc4c903feb6c: Status 404 returned error can't find the container with id 1b52d4523b0f70b5ea3ecd2c5c8a630605f719a86dffde6a6065bc4c903feb6c Jul 07 06:37:02.079233 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:02.079196 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" event={"ID":"1057d10c-fc71-480b-8f6a-1a6f4832748c","Type":"ContainerStarted","Data":"caf552aff0e97f13a398aa414804bc33eba2a4cf3225ede22b79044588668ef7"} Jul 07 06:37:02.080732 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:02.080696 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" event={"ID":"230065e6-ce67-4fbc-95bf-dff89318b04a","Type":"ContainerStarted","Data":"1b52d4523b0f70b5ea3ecd2c5c8a630605f719a86dffde6a6065bc4c903feb6c"} Jul 07 06:37:02.139637 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:02.139569 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" podStartSLOduration=1.5056787 podStartE2EDuration="4.139547762s" totalImagesPullingTime="2.633869062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:37:02.133812807 +0000 UTC m=+297.973653787" watchObservedRunningTime="2026-07-07 06:37:02.139547762 +0000 UTC m=+297.979388729" Jul 07 06:37:02.530138 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:02.530045 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:37:02.627158 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:02.627120 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:37:02.695977 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:02.695941 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:37:02.741554 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:02.741525 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:37:02.747210 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:02.747186 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:37:02.861317 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:02.861237 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:37:03.085668 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:03.085636 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:37:03.086773 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:03.086736 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf42bvt5" Jul 07 06:37:03.690263 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:03.690227 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:37:03.690565 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:03.690291 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:37:03.828054 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:03.828017 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:37:03.978128 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:03.978051 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:37:04.092913 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:04.092874 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" event={"ID":"230065e6-ce67-4fbc-95bf-dff89318b04a","Type":"ContainerStarted","Data":"128d8c25d8e609bb3cf298a161e4320656324bfbd1a18797277e0d2450eb523c"} Jul 07 06:37:04.093128 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:04.093086 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:04.095066 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:04.095042 2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-2gjqq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 06:37:04.095184 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:04.095090 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" podUID="230065e6-ce67-4fbc-95bf-dff89318b04a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 06:37:04.155693 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:04.155638 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" podStartSLOduration=2.068181813 podStartE2EDuration="4.155621762s" totalImagesPullingTime="2.087439949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:37:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:37:04.152508919 +0000 UTC m=+299.992349885" watchObservedRunningTime="2026-07-07 06:37:04.155621762 +0000 UTC m=+299.995462730" Jul 07 06:37:04.226241 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:04.226192 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:37:04.626229 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:04.626202 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 06:37:04.627401 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:04.627377 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 06:37:04.630467 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:04.630445 2579 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 06:37:05.098383 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:05.098336 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2gjqq" Jul 07 06:37:13.310733 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:13.310696 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:37:51.465476 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:51.465433 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:37:51.559769 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:51.559733 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:37:51.658827 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:51.658794 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:37:52.400169 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:52.400135 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:37:52.593094 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:52.593059 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:37:56.261313 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:37:56.261224 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:07.204270 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.204210 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx"] Jul 07 06:38:07.208003 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.207981 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" Jul 07 06:38:07.210805 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.210787 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 06:38:07.210879 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.210788 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 06:38:07.212204 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.212190 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jxfbm\"" Jul 07 06:38:07.215701 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.215675 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx"] Jul 07 06:38:07.345282 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.345249 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76847b50-0e20-4c4d-a394-7144cc003b8a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx\" (UID: \"76847b50-0e20-4c4d-a394-7144cc003b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" Jul 07 06:38:07.345479 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.345315 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76847b50-0e20-4c4d-a394-7144cc003b8a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx\" (UID: \"76847b50-0e20-4c4d-a394-7144cc003b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" Jul 07 06:38:07.345479 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.345397 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2hzr\" (UniqueName: \"kubernetes.io/projected/76847b50-0e20-4c4d-a394-7144cc003b8a-kube-api-access-s2hzr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx\" (UID: \"76847b50-0e20-4c4d-a394-7144cc003b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" Jul 07 06:38:07.446854 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.446817 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76847b50-0e20-4c4d-a394-7144cc003b8a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx\" (UID: \"76847b50-0e20-4c4d-a394-7144cc003b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" Jul 07 06:38:07.446854 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.446861 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2hzr\" (UniqueName: \"kubernetes.io/projected/76847b50-0e20-4c4d-a394-7144cc003b8a-kube-api-access-s2hzr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx\" (UID: \"76847b50-0e20-4c4d-a394-7144cc003b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" Jul 07 06:38:07.447106 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.446909 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76847b50-0e20-4c4d-a394-7144cc003b8a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx\" (UID: \"76847b50-0e20-4c4d-a394-7144cc003b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" Jul 07 06:38:07.447191 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.447169 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76847b50-0e20-4c4d-a394-7144cc003b8a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx\" (UID: \"76847b50-0e20-4c4d-a394-7144cc003b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" Jul 07 06:38:07.447246 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.447192 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76847b50-0e20-4c4d-a394-7144cc003b8a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx\" (UID: \"76847b50-0e20-4c4d-a394-7144cc003b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" Jul 07 06:38:07.455854 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.455805 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2hzr\" (UniqueName: \"kubernetes.io/projected/76847b50-0e20-4c4d-a394-7144cc003b8a-kube-api-access-s2hzr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx\" (UID: \"76847b50-0e20-4c4d-a394-7144cc003b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" Jul 07 06:38:07.517902 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.517866 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" Jul 07 06:38:07.644547 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.644519 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx"] Jul 07 06:38:07.645962 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:38:07.645934 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76847b50_0e20_4c4d_a394_7144cc003b8a.slice/crio-817e593709e3e31c8ededcf56dbff2023f2100bf7de1c9f1e83dbb34ac7e0481 WatchSource:0}: Error finding container 817e593709e3e31c8ededcf56dbff2023f2100bf7de1c9f1e83dbb34ac7e0481: Status 404 returned error can't find the container with id 817e593709e3e31c8ededcf56dbff2023f2100bf7de1c9f1e83dbb34ac7e0481 Jul 07 06:38:07.808138 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.808100 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8"] Jul 07 06:38:07.811595 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.811574 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" Jul 07 06:38:07.819791 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.819770 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8"] Jul 07 06:38:07.950968 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.950938 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j89n\" (UniqueName: \"kubernetes.io/projected/2c289d7b-c836-49e8-877e-05284e3f4d36-kube-api-access-7j89n\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8\" (UID: \"2c289d7b-c836-49e8-877e-05284e3f4d36\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" Jul 07 06:38:07.951123 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.950995 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c289d7b-c836-49e8-877e-05284e3f4d36-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8\" (UID: \"2c289d7b-c836-49e8-877e-05284e3f4d36\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" Jul 07 06:38:07.951123 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:07.951033 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c289d7b-c836-49e8-877e-05284e3f4d36-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8\" (UID: \"2c289d7b-c836-49e8-877e-05284e3f4d36\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" Jul 07 06:38:08.051821 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.051793 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c289d7b-c836-49e8-877e-05284e3f4d36-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8\" (UID: \"2c289d7b-c836-49e8-877e-05284e3f4d36\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" Jul 07 06:38:08.051976 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.051830 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c289d7b-c836-49e8-877e-05284e3f4d36-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8\" (UID: \"2c289d7b-c836-49e8-877e-05284e3f4d36\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" Jul 07 06:38:08.051976 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.051871 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j89n\" (UniqueName: \"kubernetes.io/projected/2c289d7b-c836-49e8-877e-05284e3f4d36-kube-api-access-7j89n\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8\" (UID: \"2c289d7b-c836-49e8-877e-05284e3f4d36\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" Jul 07 06:38:08.052267 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.052248 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c289d7b-c836-49e8-877e-05284e3f4d36-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8\" (UID: \"2c289d7b-c836-49e8-877e-05284e3f4d36\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" Jul 07 06:38:08.052335 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.052268 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c289d7b-c836-49e8-877e-05284e3f4d36-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8\" (UID: \"2c289d7b-c836-49e8-877e-05284e3f4d36\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" Jul 07 06:38:08.060633 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.060615 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j89n\" (UniqueName: \"kubernetes.io/projected/2c289d7b-c836-49e8-877e-05284e3f4d36-kube-api-access-7j89n\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8\" (UID: \"2c289d7b-c836-49e8-877e-05284e3f4d36\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" Jul 07 06:38:08.121735 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.121711 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" Jul 07 06:38:08.247945 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.247918 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8"] Jul 07 06:38:08.249467 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:38:08.249438 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c289d7b_c836_49e8_877e_05284e3f4d36.slice/crio-5311fb7f693956ede763b3a20d6c437003c9678f442a401a3cdb985356ef4415 WatchSource:0}: Error finding container 5311fb7f693956ede763b3a20d6c437003c9678f442a401a3cdb985356ef4415: Status 404 returned error can't find the container with id 5311fb7f693956ede763b3a20d6c437003c9678f442a401a3cdb985356ef4415 Jul 07 06:38:08.382923 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.382887 2579 generic.go:356] "Generic (PLEG): container finished" podID="2c289d7b-c836-49e8-877e-05284e3f4d36" containerID="553196ab8d79a3ec9527440b5b0a4f0864805aa37de0ee375af320a8800a8bb0" exitCode=0 Jul 07 06:38:08.383056 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.382968 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" event={"ID":"2c289d7b-c836-49e8-877e-05284e3f4d36","Type":"ContainerDied","Data":"553196ab8d79a3ec9527440b5b0a4f0864805aa37de0ee375af320a8800a8bb0"} Jul 07 06:38:08.383056 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.383028 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" event={"ID":"2c289d7b-c836-49e8-877e-05284e3f4d36","Type":"ContainerStarted","Data":"5311fb7f693956ede763b3a20d6c437003c9678f442a401a3cdb985356ef4415"} Jul 07 06:38:08.383588 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.383569 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 06:38:08.384671 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.384640 2579 generic.go:356] "Generic (PLEG): container finished" podID="76847b50-0e20-4c4d-a394-7144cc003b8a" containerID="9c6bbf4befc3436a826f2f26ac30f65503377854ed31690a32050108bb8b33fb" exitCode=0 Jul 07 06:38:08.384740 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.384722 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" event={"ID":"76847b50-0e20-4c4d-a394-7144cc003b8a","Type":"ContainerDied","Data":"9c6bbf4befc3436a826f2f26ac30f65503377854ed31690a32050108bb8b33fb"} Jul 07 06:38:08.384811 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.384767 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" event={"ID":"76847b50-0e20-4c4d-a394-7144cc003b8a","Type":"ContainerStarted","Data":"817e593709e3e31c8ededcf56dbff2023f2100bf7de1c9f1e83dbb34ac7e0481"} Jul 07 06:38:08.411889 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.411861 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5"] Jul 07 06:38:08.415269 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.415252 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" Jul 07 06:38:08.425109 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.425087 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5"] Jul 07 06:38:08.556748 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.556680 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gltc4\" (UniqueName: \"kubernetes.io/projected/530b35ad-628c-4f7a-b838-fd84a0924eec-kube-api-access-gltc4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5\" (UID: \"530b35ad-628c-4f7a-b838-fd84a0924eec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" Jul 07 06:38:08.556748 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.556730 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/530b35ad-628c-4f7a-b838-fd84a0924eec-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5\" (UID: \"530b35ad-628c-4f7a-b838-fd84a0924eec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" Jul 07 06:38:08.556922 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.556755 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/530b35ad-628c-4f7a-b838-fd84a0924eec-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5\" (UID: \"530b35ad-628c-4f7a-b838-fd84a0924eec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" Jul 07 06:38:08.658214 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.658179 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/530b35ad-628c-4f7a-b838-fd84a0924eec-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5\" (UID: \"530b35ad-628c-4f7a-b838-fd84a0924eec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" Jul 07 06:38:08.658387 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.658221 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/530b35ad-628c-4f7a-b838-fd84a0924eec-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5\" (UID: \"530b35ad-628c-4f7a-b838-fd84a0924eec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" Jul 07 06:38:08.658387 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.658270 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gltc4\" (UniqueName: \"kubernetes.io/projected/530b35ad-628c-4f7a-b838-fd84a0924eec-kube-api-access-gltc4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5\" (UID: \"530b35ad-628c-4f7a-b838-fd84a0924eec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" Jul 07 06:38:08.658620 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.658598 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/530b35ad-628c-4f7a-b838-fd84a0924eec-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5\" (UID: \"530b35ad-628c-4f7a-b838-fd84a0924eec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" Jul 07 06:38:08.658662 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.658607 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/530b35ad-628c-4f7a-b838-fd84a0924eec-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5\" (UID: \"530b35ad-628c-4f7a-b838-fd84a0924eec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" Jul 07 06:38:08.667160 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.667132 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gltc4\" (UniqueName: \"kubernetes.io/projected/530b35ad-628c-4f7a-b838-fd84a0924eec-kube-api-access-gltc4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5\" (UID: \"530b35ad-628c-4f7a-b838-fd84a0924eec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" Jul 07 06:38:08.723994 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.723967 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" Jul 07 06:38:08.810071 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.810003 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44"] Jul 07 06:38:08.813914 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.813895 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" Jul 07 06:38:08.824915 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.824887 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44"] Jul 07 06:38:08.867108 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.867085 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5"] Jul 07 06:38:08.868953 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:38:08.868925 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod530b35ad_628c_4f7a_b838_fd84a0924eec.slice/crio-760b7487409e56677b047808619ab64d063746c232cd0edf8deb387a2d173122 WatchSource:0}: Error finding container 760b7487409e56677b047808619ab64d063746c232cd0edf8deb387a2d173122: Status 404 returned error can't find the container with id 760b7487409e56677b047808619ab64d063746c232cd0edf8deb387a2d173122 Jul 07 06:38:08.961189 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.961158 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/380a39e7-ed43-402a-b786-99b16101c5d1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44\" (UID: \"380a39e7-ed43-402a-b786-99b16101c5d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" Jul 07 06:38:08.961189 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.961190 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/380a39e7-ed43-402a-b786-99b16101c5d1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44\" (UID: \"380a39e7-ed43-402a-b786-99b16101c5d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" Jul 07 06:38:08.961394 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:08.961231 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbcg6\" (UniqueName: \"kubernetes.io/projected/380a39e7-ed43-402a-b786-99b16101c5d1-kube-api-access-dbcg6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44\" (UID: \"380a39e7-ed43-402a-b786-99b16101c5d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" Jul 07 06:38:09.061880 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:09.061810 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbcg6\" (UniqueName: \"kubernetes.io/projected/380a39e7-ed43-402a-b786-99b16101c5d1-kube-api-access-dbcg6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44\" (UID: \"380a39e7-ed43-402a-b786-99b16101c5d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" Jul 07 06:38:09.062013 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:09.061882 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/380a39e7-ed43-402a-b786-99b16101c5d1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44\" (UID: \"380a39e7-ed43-402a-b786-99b16101c5d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" Jul 07 06:38:09.062013 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:09.061900 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/380a39e7-ed43-402a-b786-99b16101c5d1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44\" (UID: \"380a39e7-ed43-402a-b786-99b16101c5d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" Jul 07 06:38:09.062218 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:09.062201 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/380a39e7-ed43-402a-b786-99b16101c5d1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44\" (UID: \"380a39e7-ed43-402a-b786-99b16101c5d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" Jul 07 06:38:09.062276 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:09.062258 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/380a39e7-ed43-402a-b786-99b16101c5d1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44\" (UID: \"380a39e7-ed43-402a-b786-99b16101c5d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" Jul 07 06:38:09.070538 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:09.070516 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbcg6\" (UniqueName: \"kubernetes.io/projected/380a39e7-ed43-402a-b786-99b16101c5d1-kube-api-access-dbcg6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44\" (UID: \"380a39e7-ed43-402a-b786-99b16101c5d1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" Jul 07 06:38:09.127401 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:09.127380 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" Jul 07 06:38:09.253861 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:09.253837 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44"] Jul 07 06:38:09.255114 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:38:09.255080 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod380a39e7_ed43_402a_b786_99b16101c5d1.slice/crio-3679b83b01617bb0c975046d9190bcc77d8f3f2c87c8eb8f7529e6a523cb5406 WatchSource:0}: Error finding container 3679b83b01617bb0c975046d9190bcc77d8f3f2c87c8eb8f7529e6a523cb5406: Status 404 returned error can't find the container with id 3679b83b01617bb0c975046d9190bcc77d8f3f2c87c8eb8f7529e6a523cb5406 Jul 07 06:38:09.391072 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:09.391026 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" event={"ID":"380a39e7-ed43-402a-b786-99b16101c5d1","Type":"ContainerStarted","Data":"5cf9608a03b7aa1e6fc86e8aaa321ac38fcc444822ce5f43753b995b68b80329"} Jul 07 06:38:09.391225 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:09.391101 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" event={"ID":"380a39e7-ed43-402a-b786-99b16101c5d1","Type":"ContainerStarted","Data":"3679b83b01617bb0c975046d9190bcc77d8f3f2c87c8eb8f7529e6a523cb5406"} Jul 07 06:38:09.392618 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:09.392588 2579 generic.go:356] "Generic (PLEG): container finished" podID="530b35ad-628c-4f7a-b838-fd84a0924eec" containerID="7afe8d93c5993186b244ce20a5cebc3c53ab34baf43c388742786159801b4da3" exitCode=0 Jul 07 06:38:09.392738 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:09.392670 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" event={"ID":"530b35ad-628c-4f7a-b838-fd84a0924eec","Type":"ContainerDied","Data":"7afe8d93c5993186b244ce20a5cebc3c53ab34baf43c388742786159801b4da3"} Jul 07 06:38:09.392738 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:09.392713 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" event={"ID":"530b35ad-628c-4f7a-b838-fd84a0924eec","Type":"ContainerStarted","Data":"760b7487409e56677b047808619ab64d063746c232cd0edf8deb387a2d173122"} Jul 07 06:38:09.394538 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:09.394515 2579 generic.go:356] "Generic (PLEG): container finished" podID="2c289d7b-c836-49e8-877e-05284e3f4d36" containerID="6c55a89d58166877233e748b00edf2d40663a74eb1f4f9462491c65a2077a8ce" exitCode=0 Jul 07 06:38:09.394659 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:09.394546 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" event={"ID":"2c289d7b-c836-49e8-877e-05284e3f4d36","Type":"ContainerDied","Data":"6c55a89d58166877233e748b00edf2d40663a74eb1f4f9462491c65a2077a8ce"} Jul 07 06:38:10.402018 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:10.401990 2579 generic.go:356] "Generic (PLEG): container finished" podID="2c289d7b-c836-49e8-877e-05284e3f4d36" containerID="729edadbd72a6356332b531c5a24b65aa828f0123f327ceedf26dc67470849bf" exitCode=0 Jul 07 06:38:10.402334 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:10.402065 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" event={"ID":"2c289d7b-c836-49e8-877e-05284e3f4d36","Type":"ContainerDied","Data":"729edadbd72a6356332b531c5a24b65aa828f0123f327ceedf26dc67470849bf"} Jul 07 06:38:10.403750 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:10.403728 2579 generic.go:356] "Generic (PLEG): container finished" podID="76847b50-0e20-4c4d-a394-7144cc003b8a" containerID="ef14cbd2e167bf922e0f1cc3c12943b8d64697bbdc1f5fa9ab772b302e139fdb" exitCode=0 Jul 07 06:38:10.403842 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:10.403806 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" event={"ID":"76847b50-0e20-4c4d-a394-7144cc003b8a","Type":"ContainerDied","Data":"ef14cbd2e167bf922e0f1cc3c12943b8d64697bbdc1f5fa9ab772b302e139fdb"} Jul 07 06:38:10.405151 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:10.405131 2579 generic.go:356] "Generic (PLEG): container finished" podID="380a39e7-ed43-402a-b786-99b16101c5d1" containerID="5cf9608a03b7aa1e6fc86e8aaa321ac38fcc444822ce5f43753b995b68b80329" exitCode=0 Jul 07 06:38:10.405232 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:10.405209 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" event={"ID":"380a39e7-ed43-402a-b786-99b16101c5d1","Type":"ContainerDied","Data":"5cf9608a03b7aa1e6fc86e8aaa321ac38fcc444822ce5f43753b995b68b80329"} Jul 07 06:38:11.412589 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.412554 2579 generic.go:356] "Generic (PLEG): container finished" podID="76847b50-0e20-4c4d-a394-7144cc003b8a" containerID="3771ee97f2f0729730188c59945615397def6a704841608eb4ddae0ea0e5f38a" exitCode=0 Jul 07 06:38:11.412971 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.412637 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" event={"ID":"76847b50-0e20-4c4d-a394-7144cc003b8a","Type":"ContainerDied","Data":"3771ee97f2f0729730188c59945615397def6a704841608eb4ddae0ea0e5f38a"} Jul 07 06:38:11.414457 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.414433 2579 generic.go:356] "Generic (PLEG): container finished" podID="380a39e7-ed43-402a-b786-99b16101c5d1" containerID="6bb13c05183c1f7c835ce75236463ff576154cf3a0ba11128c4addce9b484a23" exitCode=0 Jul 07 06:38:11.414545 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.414514 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" event={"ID":"380a39e7-ed43-402a-b786-99b16101c5d1","Type":"ContainerDied","Data":"6bb13c05183c1f7c835ce75236463ff576154cf3a0ba11128c4addce9b484a23"} Jul 07 06:38:11.416492 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.416446 2579 generic.go:356] "Generic (PLEG): container finished" podID="530b35ad-628c-4f7a-b838-fd84a0924eec" containerID="50af51e6f7ad724b25859b58bf47fb48ce43d04b24a928727c7869be9e4e9af1" exitCode=0 Jul 07 06:38:11.416596 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.416514 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" event={"ID":"530b35ad-628c-4f7a-b838-fd84a0924eec","Type":"ContainerDied","Data":"50af51e6f7ad724b25859b58bf47fb48ce43d04b24a928727c7869be9e4e9af1"} Jul 07 06:38:11.551487 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.551459 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" Jul 07 06:38:11.582791 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:38:11.582760 2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod530b35ad_628c_4f7a_b838_fd84a0924eec.slice/crio-6aeab1925a9b2f37f7002ec71f016165af99cdbe80e486bfeb7492e69f915b61.scope\": RecentStats: unable to find data in memory cache]" Jul 07 06:38:11.682793 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.682761 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c289d7b-c836-49e8-877e-05284e3f4d36-bundle\") pod \"2c289d7b-c836-49e8-877e-05284e3f4d36\" (UID: \"2c289d7b-c836-49e8-877e-05284e3f4d36\") " Jul 07 06:38:11.682965 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.682829 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7j89n\" (UniqueName: \"kubernetes.io/projected/2c289d7b-c836-49e8-877e-05284e3f4d36-kube-api-access-7j89n\") pod \"2c289d7b-c836-49e8-877e-05284e3f4d36\" (UID: \"2c289d7b-c836-49e8-877e-05284e3f4d36\") " Jul 07 06:38:11.682965 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.682904 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c289d7b-c836-49e8-877e-05284e3f4d36-util\") pod \"2c289d7b-c836-49e8-877e-05284e3f4d36\" (UID: \"2c289d7b-c836-49e8-877e-05284e3f4d36\") " Jul 07 06:38:11.683485 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.683461 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c289d7b-c836-49e8-877e-05284e3f4d36-bundle" (OuterVolumeSpecName: "bundle") pod "2c289d7b-c836-49e8-877e-05284e3f4d36" (UID: "2c289d7b-c836-49e8-877e-05284e3f4d36"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:38:11.684984 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.684965 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c289d7b-c836-49e8-877e-05284e3f4d36-kube-api-access-7j89n" (OuterVolumeSpecName: "kube-api-access-7j89n") pod "2c289d7b-c836-49e8-877e-05284e3f4d36" (UID: "2c289d7b-c836-49e8-877e-05284e3f4d36"). InnerVolumeSpecName "kube-api-access-7j89n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:38:11.687637 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.687601 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c289d7b-c836-49e8-877e-05284e3f4d36-util" (OuterVolumeSpecName: "util") pod "2c289d7b-c836-49e8-877e-05284e3f4d36" (UID: "2c289d7b-c836-49e8-877e-05284e3f4d36"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:38:11.784289 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.784207 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j89n\" (UniqueName: \"kubernetes.io/projected/2c289d7b-c836-49e8-877e-05284e3f4d36-kube-api-access-7j89n\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:38:11.784289 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.784235 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c289d7b-c836-49e8-877e-05284e3f4d36-util\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:38:11.784289 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:11.784244 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c289d7b-c836-49e8-877e-05284e3f4d36-bundle\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:38:12.237719 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.237629 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:12.424021 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.423985 2579 generic.go:356] "Generic (PLEG): container finished" podID="380a39e7-ed43-402a-b786-99b16101c5d1" containerID="53d3b65b1fd7eefdcc0b79d6d93eea9d3525c68f0b108eda8c9ee6feb9649fae" exitCode=0 Jul 07 06:38:12.424475 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.424073 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" event={"ID":"380a39e7-ed43-402a-b786-99b16101c5d1","Type":"ContainerDied","Data":"53d3b65b1fd7eefdcc0b79d6d93eea9d3525c68f0b108eda8c9ee6feb9649fae"} Jul 07 06:38:12.426231 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.426203 2579 generic.go:356] "Generic (PLEG): container finished" podID="530b35ad-628c-4f7a-b838-fd84a0924eec" containerID="6aeab1925a9b2f37f7002ec71f016165af99cdbe80e486bfeb7492e69f915b61" exitCode=0 Jul 07 06:38:12.426360 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.426284 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" event={"ID":"530b35ad-628c-4f7a-b838-fd84a0924eec","Type":"ContainerDied","Data":"6aeab1925a9b2f37f7002ec71f016165af99cdbe80e486bfeb7492e69f915b61"} Jul 07 06:38:12.428144 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.428118 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" event={"ID":"2c289d7b-c836-49e8-877e-05284e3f4d36","Type":"ContainerDied","Data":"5311fb7f693956ede763b3a20d6c437003c9678f442a401a3cdb985356ef4415"} Jul 07 06:38:12.428241 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.428153 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8" Jul 07 06:38:12.428241 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.428165 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5311fb7f693956ede763b3a20d6c437003c9678f442a401a3cdb985356ef4415" Jul 07 06:38:12.558770 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.558749 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" Jul 07 06:38:12.692922 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.692890 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76847b50-0e20-4c4d-a394-7144cc003b8a-util\") pod \"76847b50-0e20-4c4d-a394-7144cc003b8a\" (UID: \"76847b50-0e20-4c4d-a394-7144cc003b8a\") " Jul 07 06:38:12.693057 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.692968 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76847b50-0e20-4c4d-a394-7144cc003b8a-bundle\") pod \"76847b50-0e20-4c4d-a394-7144cc003b8a\" (UID: \"76847b50-0e20-4c4d-a394-7144cc003b8a\") " Jul 07 06:38:12.693057 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.692990 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2hzr\" (UniqueName: \"kubernetes.io/projected/76847b50-0e20-4c4d-a394-7144cc003b8a-kube-api-access-s2hzr\") pod \"76847b50-0e20-4c4d-a394-7144cc003b8a\" (UID: \"76847b50-0e20-4c4d-a394-7144cc003b8a\") " Jul 07 06:38:12.693399 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.693339 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76847b50-0e20-4c4d-a394-7144cc003b8a-bundle" (OuterVolumeSpecName: "bundle") pod "76847b50-0e20-4c4d-a394-7144cc003b8a" (UID: "76847b50-0e20-4c4d-a394-7144cc003b8a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:38:12.695153 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.695125 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76847b50-0e20-4c4d-a394-7144cc003b8a-kube-api-access-s2hzr" (OuterVolumeSpecName: "kube-api-access-s2hzr") pod "76847b50-0e20-4c4d-a394-7144cc003b8a" (UID: "76847b50-0e20-4c4d-a394-7144cc003b8a"). InnerVolumeSpecName "kube-api-access-s2hzr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:38:12.698401 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.698357 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76847b50-0e20-4c4d-a394-7144cc003b8a-util" (OuterVolumeSpecName: "util") pod "76847b50-0e20-4c4d-a394-7144cc003b8a" (UID: "76847b50-0e20-4c4d-a394-7144cc003b8a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:38:12.794985 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.794920 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76847b50-0e20-4c4d-a394-7144cc003b8a-util\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:38:12.794985 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.794958 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76847b50-0e20-4c4d-a394-7144cc003b8a-bundle\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:38:12.794985 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:12.794968 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2hzr\" (UniqueName: \"kubernetes.io/projected/76847b50-0e20-4c4d-a394-7144cc003b8a-kube-api-access-s2hzr\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:38:13.435274 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.435243 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" Jul 07 06:38:13.435695 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.435244 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx" event={"ID":"76847b50-0e20-4c4d-a394-7144cc003b8a","Type":"ContainerDied","Data":"817e593709e3e31c8ededcf56dbff2023f2100bf7de1c9f1e83dbb34ac7e0481"} Jul 07 06:38:13.435695 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.435426 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="817e593709e3e31c8ededcf56dbff2023f2100bf7de1c9f1e83dbb34ac7e0481" Jul 07 06:38:13.586761 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.586739 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" Jul 07 06:38:13.588758 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.588739 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" Jul 07 06:38:13.646677 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.646641 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:13.703455 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.703348 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/380a39e7-ed43-402a-b786-99b16101c5d1-util\") pod \"380a39e7-ed43-402a-b786-99b16101c5d1\" (UID: \"380a39e7-ed43-402a-b786-99b16101c5d1\") " Jul 07 06:38:13.703455 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.703407 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/380a39e7-ed43-402a-b786-99b16101c5d1-bundle\") pod \"380a39e7-ed43-402a-b786-99b16101c5d1\" (UID: \"380a39e7-ed43-402a-b786-99b16101c5d1\") " Jul 07 06:38:13.703677 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.703479 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/530b35ad-628c-4f7a-b838-fd84a0924eec-util\") pod \"530b35ad-628c-4f7a-b838-fd84a0924eec\" (UID: \"530b35ad-628c-4f7a-b838-fd84a0924eec\") " Jul 07 06:38:13.703677 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.703526 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gltc4\" (UniqueName: \"kubernetes.io/projected/530b35ad-628c-4f7a-b838-fd84a0924eec-kube-api-access-gltc4\") pod \"530b35ad-628c-4f7a-b838-fd84a0924eec\" (UID: \"530b35ad-628c-4f7a-b838-fd84a0924eec\") " Jul 07 06:38:13.703677 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.703562 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/530b35ad-628c-4f7a-b838-fd84a0924eec-bundle\") pod \"530b35ad-628c-4f7a-b838-fd84a0924eec\" (UID: \"530b35ad-628c-4f7a-b838-fd84a0924eec\") " Jul 07 06:38:13.703677 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.703587 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbcg6\" (UniqueName: \"kubernetes.io/projected/380a39e7-ed43-402a-b786-99b16101c5d1-kube-api-access-dbcg6\") pod \"380a39e7-ed43-402a-b786-99b16101c5d1\" (UID: \"380a39e7-ed43-402a-b786-99b16101c5d1\") " Jul 07 06:38:13.704128 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.704080 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/380a39e7-ed43-402a-b786-99b16101c5d1-bundle" (OuterVolumeSpecName: "bundle") pod "380a39e7-ed43-402a-b786-99b16101c5d1" (UID: "380a39e7-ed43-402a-b786-99b16101c5d1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:38:13.704286 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.704224 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/530b35ad-628c-4f7a-b838-fd84a0924eec-bundle" (OuterVolumeSpecName: "bundle") pod "530b35ad-628c-4f7a-b838-fd84a0924eec" (UID: "530b35ad-628c-4f7a-b838-fd84a0924eec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:38:13.705922 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.705893 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/530b35ad-628c-4f7a-b838-fd84a0924eec-kube-api-access-gltc4" (OuterVolumeSpecName: "kube-api-access-gltc4") pod "530b35ad-628c-4f7a-b838-fd84a0924eec" (UID: "530b35ad-628c-4f7a-b838-fd84a0924eec"). InnerVolumeSpecName "kube-api-access-gltc4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:38:13.706206 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.706187 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/380a39e7-ed43-402a-b786-99b16101c5d1-kube-api-access-dbcg6" (OuterVolumeSpecName: "kube-api-access-dbcg6") pod "380a39e7-ed43-402a-b786-99b16101c5d1" (UID: "380a39e7-ed43-402a-b786-99b16101c5d1"). InnerVolumeSpecName "kube-api-access-dbcg6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:38:13.709452 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.709418 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/380a39e7-ed43-402a-b786-99b16101c5d1-util" (OuterVolumeSpecName: "util") pod "380a39e7-ed43-402a-b786-99b16101c5d1" (UID: "380a39e7-ed43-402a-b786-99b16101c5d1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:38:13.709539 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.709509 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/530b35ad-628c-4f7a-b838-fd84a0924eec-util" (OuterVolumeSpecName: "util") pod "530b35ad-628c-4f7a-b838-fd84a0924eec" (UID: "530b35ad-628c-4f7a-b838-fd84a0924eec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:38:13.711570 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.711533 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:13.804836 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.804794 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gltc4\" (UniqueName: \"kubernetes.io/projected/530b35ad-628c-4f7a-b838-fd84a0924eec-kube-api-access-gltc4\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:38:13.804836 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.804826 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/530b35ad-628c-4f7a-b838-fd84a0924eec-bundle\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:38:13.804836 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.804835 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbcg6\" (UniqueName: \"kubernetes.io/projected/380a39e7-ed43-402a-b786-99b16101c5d1-kube-api-access-dbcg6\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:38:13.804836 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.804844 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/380a39e7-ed43-402a-b786-99b16101c5d1-util\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:38:13.805081 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.804853 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/380a39e7-ed43-402a-b786-99b16101c5d1-bundle\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:38:13.805081 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:13.804860 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/530b35ad-628c-4f7a-b838-fd84a0924eec-util\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:38:14.442586 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:14.442556 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" Jul 07 06:38:14.442586 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:14.442574 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44" event={"ID":"380a39e7-ed43-402a-b786-99b16101c5d1","Type":"ContainerDied","Data":"3679b83b01617bb0c975046d9190bcc77d8f3f2c87c8eb8f7529e6a523cb5406"} Jul 07 06:38:14.443078 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:14.442626 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3679b83b01617bb0c975046d9190bcc77d8f3f2c87c8eb8f7529e6a523cb5406" Jul 07 06:38:14.444501 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:14.444466 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" event={"ID":"530b35ad-628c-4f7a-b838-fd84a0924eec","Type":"ContainerDied","Data":"760b7487409e56677b047808619ab64d063746c232cd0edf8deb387a2d173122"} Jul 07 06:38:14.444642 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:14.444505 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="760b7487409e56677b047808619ab64d063746c232cd0edf8deb387a2d173122" Jul 07 06:38:14.444642 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:14.444476 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5" Jul 07 06:38:20.874459 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:20.874425 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:24.135038 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:24.134997 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:24.233645 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:24.233611 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:24.331348 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:24.331317 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:24.427049 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:24.426969 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:25.102020 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:25.101988 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:27.526677 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:27.526640 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:27.668189 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:27.668154 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:28.905205 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:28.905157 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:32.070704 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:32.070671 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:32.500188 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:32.500106 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:32.733866 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:32.733832 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:35.504706 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:35.504669 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:36.898666 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:36.898625 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:37.001625 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:37.001589 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:40.663577 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:40.663540 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:40.769273 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:40.769232 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:41.782795 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:41.782767 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:41.882399 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:41.882319 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:44.466526 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:44.466491 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:44.567562 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:44.567527 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:52.935880 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:52.935842 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:53.015645 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:53.015611 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:55.458385 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:55.458341 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:55.567259 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:55.567223 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:58.272571 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:58.272533 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:38:58.374851 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:38:58.374820 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:01.849728 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:01.849694 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:01.950328 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:01.950296 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:10.664262 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:10.664229 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:10.764630 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:10.764599 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:13.267732 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:13.267693 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:13.367908 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:13.367879 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:16.467826 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:16.467790 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:16.589718 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:16.589684 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:19.876237 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:19.876198 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:19.976561 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:19.976528 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:21.083158 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083123 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg"] Jul 07 06:39:21.083626 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083610 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c289d7b-c836-49e8-877e-05284e3f4d36" containerName="util" Jul 07 06:39:21.083692 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083629 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c289d7b-c836-49e8-877e-05284e3f4d36" containerName="util" Jul 07 06:39:21.083692 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083642 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c289d7b-c836-49e8-877e-05284e3f4d36" containerName="pull" Jul 07 06:39:21.083692 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083650 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c289d7b-c836-49e8-877e-05284e3f4d36" containerName="pull" Jul 07 06:39:21.083692 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083665 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76847b50-0e20-4c4d-a394-7144cc003b8a" containerName="extract" Jul 07 06:39:21.083692 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083676 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="76847b50-0e20-4c4d-a394-7144cc003b8a" containerName="extract" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083700 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76847b50-0e20-4c4d-a394-7144cc003b8a" containerName="pull" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083708 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="76847b50-0e20-4c4d-a394-7144cc003b8a" containerName="pull" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083720 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="530b35ad-628c-4f7a-b838-fd84a0924eec" containerName="pull" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083728 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="530b35ad-628c-4f7a-b838-fd84a0924eec" containerName="pull" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083741 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76847b50-0e20-4c4d-a394-7144cc003b8a" containerName="util" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083749 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="76847b50-0e20-4c4d-a394-7144cc003b8a" containerName="util" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083758 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="380a39e7-ed43-402a-b786-99b16101c5d1" containerName="pull" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083766 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="380a39e7-ed43-402a-b786-99b16101c5d1" containerName="pull" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083778 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="530b35ad-628c-4f7a-b838-fd84a0924eec" containerName="util" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083786 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="530b35ad-628c-4f7a-b838-fd84a0924eec" containerName="util" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083796 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="380a39e7-ed43-402a-b786-99b16101c5d1" containerName="extract" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083804 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="380a39e7-ed43-402a-b786-99b16101c5d1" containerName="extract" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083820 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c289d7b-c836-49e8-877e-05284e3f4d36" containerName="extract" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083828 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c289d7b-c836-49e8-877e-05284e3f4d36" containerName="extract" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083842 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="380a39e7-ed43-402a-b786-99b16101c5d1" containerName="util" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083850 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="380a39e7-ed43-402a-b786-99b16101c5d1" containerName="util" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083861 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="530b35ad-628c-4f7a-b838-fd84a0924eec" containerName="extract" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083870 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="530b35ad-628c-4f7a-b838-fd84a0924eec" containerName="extract" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083941 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="530b35ad-628c-4f7a-b838-fd84a0924eec" containerName="extract" Jul 07 06:39:21.083942 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083951 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c289d7b-c836-49e8-877e-05284e3f4d36" containerName="extract" Jul 07 06:39:21.084895 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083962 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="76847b50-0e20-4c4d-a394-7144cc003b8a" containerName="extract" Jul 07 06:39:21.084895 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.083971 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="380a39e7-ed43-402a-b786-99b16101c5d1" containerName="extract" Jul 07 06:39:21.087333 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.087314 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.090050 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.090034 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-79r4g\"" Jul 07 06:39:21.100771 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.100749 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg"] Jul 07 06:39:21.114495 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.114469 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:21.211524 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.211491 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:21.256013 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.255984 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q8ff\" (UniqueName: \"kubernetes.io/projected/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-kube-api-access-2q8ff\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.256013 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.256017 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.256224 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.256039 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.256224 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.256070 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.256224 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.256094 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.256224 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.256109 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.256224 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.256132 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.256224 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.256198 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.256468 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.256239 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.357544 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.357457 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q8ff\" (UniqueName: \"kubernetes.io/projected/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-kube-api-access-2q8ff\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.357544 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.357498 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.357544 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.357520 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.357544 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.357541 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.357869 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.357560 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.357869 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.357687 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.357869 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.357740 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.357869 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.357780 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.357869 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.357829 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.358129 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.357911 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.358129 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.357958 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.358129 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.358122 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.358352 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.358199 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.358525 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.358506 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.359884 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.359861 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.360070 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.360053 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.370163 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.370136 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q8ff\" (UniqueName: \"kubernetes.io/projected/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-kube-api-access-2q8ff\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.370514 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.370497 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b917a0bf-164b-40e9-88a6-ad5ac62e29ca-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-rrfhg\" (UID: \"b917a0bf-164b-40e9-88a6-ad5ac62e29ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.397582 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.397548 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:21.536174 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.536144 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg"] Jul 07 06:39:21.537658 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:39:21.537631 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb917a0bf_164b_40e9_88a6_ad5ac62e29ca.slice/crio-fd3cec21b247165e42a1e096cfb87bd9ec585247e7eec09eb0bf1c5da769fbc6 WatchSource:0}: Error finding container fd3cec21b247165e42a1e096cfb87bd9ec585247e7eec09eb0bf1c5da769fbc6: Status 404 returned error can't find the container with id fd3cec21b247165e42a1e096cfb87bd9ec585247e7eec09eb0bf1c5da769fbc6 Jul 07 06:39:21.539616 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.539589 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:39:21.539666 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.539652 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:39:21.539706 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.539680 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:39:21.768223 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.768131 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" event={"ID":"b917a0bf-164b-40e9-88a6-ad5ac62e29ca","Type":"ContainerStarted","Data":"f799da803b240abae588f6fb1559c2cb74c38653477eb7e80b12f95e22a04f9f"} Jul 07 06:39:21.768223 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.768203 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" event={"ID":"b917a0bf-164b-40e9-88a6-ad5ac62e29ca","Type":"ContainerStarted","Data":"fd3cec21b247165e42a1e096cfb87bd9ec585247e7eec09eb0bf1c5da769fbc6"} Jul 07 06:39:21.789636 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:21.789579 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" podStartSLOduration=0.789562044 podStartE2EDuration="789.562044ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:39:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:39:21.787717401 +0000 UTC m=+437.627558369" watchObservedRunningTime="2026-07-07 06:39:21.789562044 +0000 UTC m=+437.629403023" Jul 07 06:39:22.398239 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:22.398191 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:22.402981 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:22.402958 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:22.773902 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:22.773821 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:22.774860 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:22.774841 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:39:28.831517 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:28.831432 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:28.886280 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:28.886241 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:30.029342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:30.029304 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:30.084615 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:30.084581 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:35.634327 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:35.634295 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:35.744829 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:35.744790 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:35.803277 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:35.803242 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:36.034258 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:36.034225 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:36.138541 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:36.138504 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:36.460290 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:36.460207 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:36.534184 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:36.534146 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:54.540607 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:54.540568 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:54.621549 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:54.621516 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:39:54.678228 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:39:54.678195 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:06.759948 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:06.759909 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:07.109552 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:07.109515 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:07.332764 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:07.332726 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:12.937523 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:12.937484 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:12.949656 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:12.949625 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-768ccc9575-jgsrd"] Jul 07 06:40:12.952982 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:12.952961 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-768ccc9575-jgsrd" Jul 07 06:40:12.956253 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:12.956230 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 06:40:12.956482 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:12.956466 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-scvt9\"" Jul 07 06:40:12.956794 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:12.956781 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 06:40:12.969738 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:12.969712 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-768ccc9575-jgsrd"] Jul 07 06:40:13.020733 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:13.020696 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjm7w\" (UniqueName: \"kubernetes.io/projected/61fa8af6-9406-4c82-a7f4-d53871e789e2-kube-api-access-cjm7w\") pod \"maas-controller-768ccc9575-jgsrd\" (UID: \"61fa8af6-9406-4c82-a7f4-d53871e789e2\") " pod="opendatahub/maas-controller-768ccc9575-jgsrd" Jul 07 06:40:13.020892 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:13.020768 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61fa8af6-9406-4c82-a7f4-d53871e789e2-webhook-cert\") pod \"maas-controller-768ccc9575-jgsrd\" (UID: \"61fa8af6-9406-4c82-a7f4-d53871e789e2\") " pod="opendatahub/maas-controller-768ccc9575-jgsrd" Jul 07 06:40:13.036755 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:13.036726 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:13.097250 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:13.097222 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:13.121590 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:13.121560 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61fa8af6-9406-4c82-a7f4-d53871e789e2-webhook-cert\") pod \"maas-controller-768ccc9575-jgsrd\" (UID: \"61fa8af6-9406-4c82-a7f4-d53871e789e2\") " pod="opendatahub/maas-controller-768ccc9575-jgsrd" Jul 07 06:40:13.121731 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:13.121640 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjm7w\" (UniqueName: \"kubernetes.io/projected/61fa8af6-9406-4c82-a7f4-d53871e789e2-kube-api-access-cjm7w\") pod \"maas-controller-768ccc9575-jgsrd\" (UID: \"61fa8af6-9406-4c82-a7f4-d53871e789e2\") " pod="opendatahub/maas-controller-768ccc9575-jgsrd" Jul 07 06:40:13.121731 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:40:13.121703 2579 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 07 06:40:13.121843 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:40:13.121772 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61fa8af6-9406-4c82-a7f4-d53871e789e2-webhook-cert podName:61fa8af6-9406-4c82-a7f4-d53871e789e2 nodeName:}" failed. No retries permitted until 2026-07-07 06:40:13.621756899 +0000 UTC m=+489.461597844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/61fa8af6-9406-4c82-a7f4-d53871e789e2-webhook-cert") pod "maas-controller-768ccc9575-jgsrd" (UID: "61fa8af6-9406-4c82-a7f4-d53871e789e2") : secret "maas-controller-webhook-cert" not found Jul 07 06:40:13.133357 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:13.133326 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjm7w\" (UniqueName: \"kubernetes.io/projected/61fa8af6-9406-4c82-a7f4-d53871e789e2-kube-api-access-cjm7w\") pod \"maas-controller-768ccc9575-jgsrd\" (UID: \"61fa8af6-9406-4c82-a7f4-d53871e789e2\") " pod="opendatahub/maas-controller-768ccc9575-jgsrd" Jul 07 06:40:13.625316 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:13.625278 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61fa8af6-9406-4c82-a7f4-d53871e789e2-webhook-cert\") pod \"maas-controller-768ccc9575-jgsrd\" (UID: \"61fa8af6-9406-4c82-a7f4-d53871e789e2\") " pod="opendatahub/maas-controller-768ccc9575-jgsrd" Jul 07 06:40:13.625530 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:40:13.625458 2579 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 07 06:40:13.625530 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:40:13.625525 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61fa8af6-9406-4c82-a7f4-d53871e789e2-webhook-cert podName:61fa8af6-9406-4c82-a7f4-d53871e789e2 nodeName:}" failed. No retries permitted until 2026-07-07 06:40:14.625509143 +0000 UTC m=+490.465350088 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/61fa8af6-9406-4c82-a7f4-d53871e789e2-webhook-cert") pod "maas-controller-768ccc9575-jgsrd" (UID: "61fa8af6-9406-4c82-a7f4-d53871e789e2") : secret "maas-controller-webhook-cert" not found Jul 07 06:40:13.872292 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:13.872260 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:14.635770 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:14.635735 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61fa8af6-9406-4c82-a7f4-d53871e789e2-webhook-cert\") pod \"maas-controller-768ccc9575-jgsrd\" (UID: \"61fa8af6-9406-4c82-a7f4-d53871e789e2\") " pod="opendatahub/maas-controller-768ccc9575-jgsrd" Jul 07 06:40:14.638144 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:14.638123 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61fa8af6-9406-4c82-a7f4-d53871e789e2-webhook-cert\") pod \"maas-controller-768ccc9575-jgsrd\" (UID: \"61fa8af6-9406-4c82-a7f4-d53871e789e2\") " pod="opendatahub/maas-controller-768ccc9575-jgsrd" Jul 07 06:40:14.762693 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:14.762660 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-768ccc9575-jgsrd" Jul 07 06:40:14.891795 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:14.891766 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-768ccc9575-jgsrd"] Jul 07 06:40:14.896751 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:40:14.894156 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61fa8af6_9406_4c82_a7f4_d53871e789e2.slice/crio-ca2059c322a2d7b0176d45331b9f51593314c109367ab5b0f61b4ec390a09934 WatchSource:0}: Error finding container ca2059c322a2d7b0176d45331b9f51593314c109367ab5b0f61b4ec390a09934: Status 404 returned error can't find the container with id ca2059c322a2d7b0176d45331b9f51593314c109367ab5b0f61b4ec390a09934 Jul 07 06:40:15.028738 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:15.028703 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-768ccc9575-jgsrd" event={"ID":"61fa8af6-9406-4c82-a7f4-d53871e789e2","Type":"ContainerStarted","Data":"ca2059c322a2d7b0176d45331b9f51593314c109367ab5b0f61b4ec390a09934"} Jul 07 06:40:16.731166 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:16.731119 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:16.830152 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:16.830129 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:17.042431 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:17.042398 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-768ccc9575-jgsrd" event={"ID":"61fa8af6-9406-4c82-a7f4-d53871e789e2","Type":"ContainerStarted","Data":"3f5667ce53fd85f772d68abff3a93eb0e579a58d7daa61cefcd04032df87d5b7"} Jul 07 06:40:17.042599 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:17.042498 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-768ccc9575-jgsrd" Jul 07 06:40:17.091349 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:17.091295 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-768ccc9575-jgsrd" podStartSLOduration=3.219579815 podStartE2EDuration="5.091276104s" totalImagesPullingTime="1.871696289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:40:17.090206663 +0000 UTC m=+492.930047631" watchObservedRunningTime="2026-07-07 06:40:17.091276104 +0000 UTC m=+492.931117072" Jul 07 06:40:28.053914 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:28.053884 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-768ccc9575-jgsrd" Jul 07 06:40:28.558969 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:28.558936 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-768ccc9575-jgsrd"] Jul 07 06:40:28.559153 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:28.559133 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-768ccc9575-jgsrd" podUID="61fa8af6-9406-4c82-a7f4-d53871e789e2" containerName="manager" containerID="cri-o://3f5667ce53fd85f772d68abff3a93eb0e579a58d7daa61cefcd04032df87d5b7" gracePeriod=30 Jul 07 06:40:28.800721 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:28.800691 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-768ccc9575-jgsrd" Jul 07 06:40:28.865789 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:28.865711 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjm7w\" (UniqueName: \"kubernetes.io/projected/61fa8af6-9406-4c82-a7f4-d53871e789e2-kube-api-access-cjm7w\") pod \"61fa8af6-9406-4c82-a7f4-d53871e789e2\" (UID: \"61fa8af6-9406-4c82-a7f4-d53871e789e2\") " Jul 07 06:40:28.865951 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:28.865845 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61fa8af6-9406-4c82-a7f4-d53871e789e2-webhook-cert\") pod \"61fa8af6-9406-4c82-a7f4-d53871e789e2\" (UID: \"61fa8af6-9406-4c82-a7f4-d53871e789e2\") " Jul 07 06:40:28.867797 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:28.867766 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61fa8af6-9406-4c82-a7f4-d53871e789e2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "61fa8af6-9406-4c82-a7f4-d53871e789e2" (UID: "61fa8af6-9406-4c82-a7f4-d53871e789e2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 06:40:28.867907 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:28.867837 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61fa8af6-9406-4c82-a7f4-d53871e789e2-kube-api-access-cjm7w" (OuterVolumeSpecName: "kube-api-access-cjm7w") pod "61fa8af6-9406-4c82-a7f4-d53871e789e2" (UID: "61fa8af6-9406-4c82-a7f4-d53871e789e2"). InnerVolumeSpecName "kube-api-access-cjm7w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:40:28.967407 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:28.967350 2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61fa8af6-9406-4c82-a7f4-d53871e789e2-webhook-cert\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:40:28.967407 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:28.967400 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjm7w\" (UniqueName: \"kubernetes.io/projected/61fa8af6-9406-4c82-a7f4-d53871e789e2-kube-api-access-cjm7w\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:40:29.105614 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:29.105578 2579 generic.go:356] "Generic (PLEG): container finished" podID="61fa8af6-9406-4c82-a7f4-d53871e789e2" containerID="3f5667ce53fd85f772d68abff3a93eb0e579a58d7daa61cefcd04032df87d5b7" exitCode=0 Jul 07 06:40:29.105986 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:29.105624 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-768ccc9575-jgsrd" Jul 07 06:40:29.105986 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:29.105656 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-768ccc9575-jgsrd" event={"ID":"61fa8af6-9406-4c82-a7f4-d53871e789e2","Type":"ContainerDied","Data":"3f5667ce53fd85f772d68abff3a93eb0e579a58d7daa61cefcd04032df87d5b7"} Jul 07 06:40:29.105986 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:29.105713 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-768ccc9575-jgsrd" event={"ID":"61fa8af6-9406-4c82-a7f4-d53871e789e2","Type":"ContainerDied","Data":"ca2059c322a2d7b0176d45331b9f51593314c109367ab5b0f61b4ec390a09934"} Jul 07 06:40:29.105986 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:29.105738 2579 scope.go:117] "RemoveContainer" containerID="3f5667ce53fd85f772d68abff3a93eb0e579a58d7daa61cefcd04032df87d5b7" Jul 07 06:40:29.117207 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:29.117188 2579 scope.go:117] "RemoveContainer" containerID="3f5667ce53fd85f772d68abff3a93eb0e579a58d7daa61cefcd04032df87d5b7" Jul 07 06:40:29.117526 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:40:29.117505 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f5667ce53fd85f772d68abff3a93eb0e579a58d7daa61cefcd04032df87d5b7\": container with ID starting with 3f5667ce53fd85f772d68abff3a93eb0e579a58d7daa61cefcd04032df87d5b7 not found: ID does not exist" containerID="3f5667ce53fd85f772d68abff3a93eb0e579a58d7daa61cefcd04032df87d5b7" Jul 07 06:40:29.117603 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:29.117534 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f5667ce53fd85f772d68abff3a93eb0e579a58d7daa61cefcd04032df87d5b7"} err="failed to get container status \"3f5667ce53fd85f772d68abff3a93eb0e579a58d7daa61cefcd04032df87d5b7\": rpc error: code = NotFound desc = could not find container \"3f5667ce53fd85f772d68abff3a93eb0e579a58d7daa61cefcd04032df87d5b7\": container with ID starting with 3f5667ce53fd85f772d68abff3a93eb0e579a58d7daa61cefcd04032df87d5b7 not found: ID does not exist" Jul 07 06:40:29.135024 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:29.134995 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-768ccc9575-jgsrd"] Jul 07 06:40:29.139792 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:29.139771 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-768ccc9575-jgsrd"] Jul 07 06:40:30.783032 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:30.782998 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61fa8af6-9406-4c82-a7f4-d53871e789e2" path="/var/lib/kubelet/pods/61fa8af6-9406-4c82-a7f4-d53871e789e2/volumes" Jul 07 06:40:48.180593 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.180557 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:48.611785 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.611749 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-858ccdc565-w6cgq"] Jul 07 06:40:48.612170 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.612151 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61fa8af6-9406-4c82-a7f4-d53871e789e2" containerName="manager" Jul 07 06:40:48.612254 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.612173 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="61fa8af6-9406-4c82-a7f4-d53871e789e2" containerName="manager" Jul 07 06:40:48.612306 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.612268 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="61fa8af6-9406-4c82-a7f4-d53871e789e2" containerName="manager" Jul 07 06:40:48.618538 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.618505 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-858ccdc565-w6cgq" Jul 07 06:40:48.619994 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.619969 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:48.621349 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.621330 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 07 06:40:48.624665 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.624644 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-858ccdc565-w6cgq"] Jul 07 06:40:48.635663 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.635481 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b63629-9690-44c7-8263-74cba91fbbfb-openshift-service-ca\") pod \"maas-api-858ccdc565-w6cgq\" (UID: \"75b63629-9690-44c7-8263-74cba91fbbfb\") " pod="opendatahub/maas-api-858ccdc565-w6cgq" Jul 07 06:40:48.635663 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.635525 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b63629-9690-44c7-8263-74cba91fbbfb-maas-api-tls\") pod \"maas-api-858ccdc565-w6cgq\" (UID: \"75b63629-9690-44c7-8263-74cba91fbbfb\") " pod="opendatahub/maas-api-858ccdc565-w6cgq" Jul 07 06:40:48.635663 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.635555 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvbdj\" (UniqueName: \"kubernetes.io/projected/75b63629-9690-44c7-8263-74cba91fbbfb-kube-api-access-xvbdj\") pod \"maas-api-858ccdc565-w6cgq\" (UID: \"75b63629-9690-44c7-8263-74cba91fbbfb\") " pod="opendatahub/maas-api-858ccdc565-w6cgq" Jul 07 06:40:48.736742 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.736709 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b63629-9690-44c7-8263-74cba91fbbfb-openshift-service-ca\") pod \"maas-api-858ccdc565-w6cgq\" (UID: \"75b63629-9690-44c7-8263-74cba91fbbfb\") " pod="opendatahub/maas-api-858ccdc565-w6cgq" Jul 07 06:40:48.736898 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.736745 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b63629-9690-44c7-8263-74cba91fbbfb-maas-api-tls\") pod \"maas-api-858ccdc565-w6cgq\" (UID: \"75b63629-9690-44c7-8263-74cba91fbbfb\") " pod="opendatahub/maas-api-858ccdc565-w6cgq" Jul 07 06:40:48.736898 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.736766 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvbdj\" (UniqueName: \"kubernetes.io/projected/75b63629-9690-44c7-8263-74cba91fbbfb-kube-api-access-xvbdj\") pod \"maas-api-858ccdc565-w6cgq\" (UID: \"75b63629-9690-44c7-8263-74cba91fbbfb\") " pod="opendatahub/maas-api-858ccdc565-w6cgq" Jul 07 06:40:48.736898 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:40:48.736877 2579 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 07 06:40:48.737078 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:40:48.736947 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75b63629-9690-44c7-8263-74cba91fbbfb-maas-api-tls podName:75b63629-9690-44c7-8263-74cba91fbbfb nodeName:}" failed. No retries permitted until 2026-07-07 06:40:49.236930634 +0000 UTC m=+525.076771578 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/75b63629-9690-44c7-8263-74cba91fbbfb-maas-api-tls") pod "maas-api-858ccdc565-w6cgq" (UID: "75b63629-9690-44c7-8263-74cba91fbbfb") : secret "maas-api-serving-cert" not found Jul 07 06:40:48.737518 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.737495 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b63629-9690-44c7-8263-74cba91fbbfb-openshift-service-ca\") pod \"maas-api-858ccdc565-w6cgq\" (UID: \"75b63629-9690-44c7-8263-74cba91fbbfb\") " pod="opendatahub/maas-api-858ccdc565-w6cgq" Jul 07 06:40:48.747643 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.747624 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvbdj\" (UniqueName: \"kubernetes.io/projected/75b63629-9690-44c7-8263-74cba91fbbfb-kube-api-access-xvbdj\") pod \"maas-api-858ccdc565-w6cgq\" (UID: \"75b63629-9690-44c7-8263-74cba91fbbfb\") " pod="opendatahub/maas-api-858ccdc565-w6cgq" Jul 07 06:40:48.795258 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:48.795199 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:49.004687 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:49.004610 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:49.110530 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:49.110502 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:49.241753 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:49.241724 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b63629-9690-44c7-8263-74cba91fbbfb-maas-api-tls\") pod \"maas-api-858ccdc565-w6cgq\" (UID: \"75b63629-9690-44c7-8263-74cba91fbbfb\") " pod="opendatahub/maas-api-858ccdc565-w6cgq" Jul 07 06:40:49.244514 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:49.244486 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b63629-9690-44c7-8263-74cba91fbbfb-maas-api-tls\") pod \"maas-api-858ccdc565-w6cgq\" (UID: \"75b63629-9690-44c7-8263-74cba91fbbfb\") " pod="opendatahub/maas-api-858ccdc565-w6cgq" Jul 07 06:40:49.530535 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:49.530500 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-858ccdc565-w6cgq" Jul 07 06:40:49.659625 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:49.659599 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-858ccdc565-w6cgq"] Jul 07 06:40:49.661397 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:40:49.661351 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75b63629_9690_44c7_8263_74cba91fbbfb.slice/crio-af19ae8fc59754ac5cfb62741fb3027ac2e9f8787e173491f3defc04ad2435a7 WatchSource:0}: Error finding container af19ae8fc59754ac5cfb62741fb3027ac2e9f8787e173491f3defc04ad2435a7: Status 404 returned error can't find the container with id af19ae8fc59754ac5cfb62741fb3027ac2e9f8787e173491f3defc04ad2435a7 Jul 07 06:40:50.208012 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:50.207976 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-858ccdc565-w6cgq" event={"ID":"75b63629-9690-44c7-8263-74cba91fbbfb","Type":"ContainerStarted","Data":"af19ae8fc59754ac5cfb62741fb3027ac2e9f8787e173491f3defc04ad2435a7"} Jul 07 06:40:52.218691 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:52.218600 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-858ccdc565-w6cgq" event={"ID":"75b63629-9690-44c7-8263-74cba91fbbfb","Type":"ContainerStarted","Data":"6c9fa4ed4114347b780760edae1aa2357ba38c5134bfa55baef05f2c0786d427"} Jul 07 06:40:52.219071 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:52.218738 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-858ccdc565-w6cgq" Jul 07 06:40:52.238875 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:52.238796 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-858ccdc565-w6cgq" podStartSLOduration=2.092126841 podStartE2EDuration="4.238773732s" totalImagesPullingTime="2.146646891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:40:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:40:52.235418673 +0000 UTC m=+528.075259663" watchObservedRunningTime="2026-07-07 06:40:52.238773732 +0000 UTC m=+528.078614697" Jul 07 06:40:57.244383 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:57.244334 2579 generic.go:356] "Generic (PLEG): container finished" podID="b917a0bf-164b-40e9-88a6-ad5ac62e29ca" containerID="f799da803b240abae588f6fb1559c2cb74c38653477eb7e80b12f95e22a04f9f" exitCode=0 Jul 07 06:40:57.244902 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:57.244412 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" event={"ID":"b917a0bf-164b-40e9-88a6-ad5ac62e29ca","Type":"ContainerDied","Data":"f799da803b240abae588f6fb1559c2cb74c38653477eb7e80b12f95e22a04f9f"} Jul 07 06:40:57.244902 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:57.244895 2579 scope.go:117] "RemoveContainer" containerID="f799da803b240abae588f6fb1559c2cb74c38653477eb7e80b12f95e22a04f9f" Jul 07 06:40:57.245890 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:57.245863 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:40:57.245984 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:57.245941 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:40:57.246044 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:57.245990 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:40:58.232717 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:58.232677 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-858ccdc565-w6cgq" Jul 07 06:40:58.255760 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:58.255720 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" event={"ID":"b917a0bf-164b-40e9-88a6-ad5ac62e29ca","Type":"ContainerStarted","Data":"983c50ebc436cda0ff00111962e7aa8e8ebf39cd0ebcd24e8e5a481e86a7947d"} Jul 07 06:40:58.398150 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:58.398114 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:40:58.398400 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:58.398356 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rrfhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 07 06:40:58.398458 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:58.398435 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" podUID="b917a0bf-164b-40e9-88a6-ad5ac62e29ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 07 06:40:59.314887 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:59.314851 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:40:59.398500 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:59.398460 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rrfhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 07 06:40:59.398699 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:59.398524 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" podUID="b917a0bf-164b-40e9-88a6-ad5ac62e29ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 07 06:40:59.448398 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:40:59.448344 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:41:00.398433 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:00.398396 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rrfhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 07 06:41:00.398907 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:00.398476 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" podUID="b917a0bf-164b-40e9-88a6-ad5ac62e29ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 07 06:41:01.398823 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:01.398791 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rrfhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 07 06:41:01.399207 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:01.398846 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" podUID="b917a0bf-164b-40e9-88a6-ad5ac62e29ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 07 06:41:02.401253 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.401225 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:41:02.401644 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.401498 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:41:02.402132 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.402115 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rrfhg" Jul 07 06:41:02.469194 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.469161 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc"] Jul 07 06:41:02.473092 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.473064 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.476099 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.476047 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 06:41:02.476237 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.476047 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 07 06:41:02.476310 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.476254 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 06:41:02.476310 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.476067 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7scrk\"" Jul 07 06:41:02.482497 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.482472 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc"] Jul 07 06:41:02.563580 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.563546 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mnt8\" (UniqueName: \"kubernetes.io/projected/67945122-a2c9-49e3-a6b1-979ce437c9fe-kube-api-access-9mnt8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.563738 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.563612 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67945122-a2c9-49e3-a6b1-979ce437c9fe-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.563738 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.563657 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67945122-a2c9-49e3-a6b1-979ce437c9fe-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.563738 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.563686 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67945122-a2c9-49e3-a6b1-979ce437c9fe-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.563902 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.563758 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67945122-a2c9-49e3-a6b1-979ce437c9fe-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.563902 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.563782 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67945122-a2c9-49e3-a6b1-979ce437c9fe-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.572745 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.572719 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:41:02.664690 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.664603 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67945122-a2c9-49e3-a6b1-979ce437c9fe-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.664690 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.664656 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67945122-a2c9-49e3-a6b1-979ce437c9fe-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.664690 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.664683 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67945122-a2c9-49e3-a6b1-979ce437c9fe-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.664959 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.664778 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67945122-a2c9-49e3-a6b1-979ce437c9fe-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.664959 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.664807 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67945122-a2c9-49e3-a6b1-979ce437c9fe-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.664959 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.664867 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mnt8\" (UniqueName: \"kubernetes.io/projected/67945122-a2c9-49e3-a6b1-979ce437c9fe-kube-api-access-9mnt8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.665295 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.665243 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67945122-a2c9-49e3-a6b1-979ce437c9fe-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.665444 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.665307 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67945122-a2c9-49e3-a6b1-979ce437c9fe-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.665444 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.665316 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67945122-a2c9-49e3-a6b1-979ce437c9fe-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.667006 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.666983 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67945122-a2c9-49e3-a6b1-979ce437c9fe-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.667269 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.667254 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67945122-a2c9-49e3-a6b1-979ce437c9fe-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.674494 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.674474 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mnt8\" (UniqueName: \"kubernetes.io/projected/67945122-a2c9-49e3-a6b1-979ce437c9fe-kube-api-access-9mnt8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc\" (UID: \"67945122-a2c9-49e3-a6b1-979ce437c9fe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.787076 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.787046 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:02.919438 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:02.919412 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc"] Jul 07 06:41:02.920771 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:41:02.920742 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67945122_a2c9_49e3_a6b1_979ce437c9fe.slice/crio-ac6df5efa8bf663ee27f3fddca301f134d14af584007702492db484961126675 WatchSource:0}: Error finding container ac6df5efa8bf663ee27f3fddca301f134d14af584007702492db484961126675: Status 404 returned error can't find the container with id ac6df5efa8bf663ee27f3fddca301f134d14af584007702492db484961126675 Jul 07 06:41:03.292930 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:03.292857 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" event={"ID":"67945122-a2c9-49e3-a6b1-979ce437c9fe","Type":"ContainerStarted","Data":"ac6df5efa8bf663ee27f3fddca301f134d14af584007702492db484961126675"} Jul 07 06:41:04.299113 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:04.299078 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" event={"ID":"67945122-a2c9-49e3-a6b1-979ce437c9fe","Type":"ContainerStarted","Data":"7b5772bd25a408a1c36e87c4452e615a874333f05210657263f80d24d6eddc50"} Jul 07 06:41:04.322185 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:04.322132 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" podStartSLOduration=1.322702104 podStartE2EDuration="2.322115757s" totalImagesPullingTime="999.413653ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:41:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:41:04.320883596 +0000 UTC m=+540.160724564" watchObservedRunningTime="2026-07-07 06:41:04.322115757 +0000 UTC m=+540.161956723" Jul 07 06:41:12.694609 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:12.694568 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:41:12.787956 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:12.787923 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:12.788113 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:12.787986 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:12.801852 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:12.801828 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:13.357169 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:13.357143 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc" Jul 07 06:41:30.306562 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:30.306528 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:41:33.829246 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:33.829213 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:41:39.321352 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:39.321307 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:41:49.517982 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:41:49.517948 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:42:04.660052 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:42:04.660023 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 06:42:04.661255 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:42:04.661232 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 06:42:34.450091 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:42:34.450014 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:42:58.582683 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:42:58.582641 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:42:59.326809 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:42:59.326775 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:43:10.263566 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:43:10.263525 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:43:10.362063 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:43:10.362026 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:43:11.421807 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:43:11.421753 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:43:11.522956 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:43:11.522924 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:44:22.130823 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:22.130787 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:44:23.690723 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:23.690685 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:44:23.804335 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:23.804298 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77b55b5c58-wt88z"] Jul 07 06:44:23.807714 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:23.807693 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77b55b5c58-wt88z" Jul 07 06:44:23.810359 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:23.810339 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 06:44:23.810689 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:23.810671 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 06:44:23.811686 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:23.811667 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-scvt9\"" Jul 07 06:44:23.821673 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:23.821654 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77b55b5c58-wt88z"] Jul 07 06:44:23.924446 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:23.924419 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7a16f5b6-099a-4204-8c82-d28b8a48732c-webhook-cert\") pod \"maas-controller-77b55b5c58-wt88z\" (UID: \"7a16f5b6-099a-4204-8c82-d28b8a48732c\") " pod="opendatahub/maas-controller-77b55b5c58-wt88z" Jul 07 06:44:23.924592 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:23.924485 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbsc6\" (UniqueName: \"kubernetes.io/projected/7a16f5b6-099a-4204-8c82-d28b8a48732c-kube-api-access-mbsc6\") pod \"maas-controller-77b55b5c58-wt88z\" (UID: \"7a16f5b6-099a-4204-8c82-d28b8a48732c\") " pod="opendatahub/maas-controller-77b55b5c58-wt88z" Jul 07 06:44:24.025184 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:24.025102 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7a16f5b6-099a-4204-8c82-d28b8a48732c-webhook-cert\") pod \"maas-controller-77b55b5c58-wt88z\" (UID: \"7a16f5b6-099a-4204-8c82-d28b8a48732c\") " pod="opendatahub/maas-controller-77b55b5c58-wt88z" Jul 07 06:44:24.025184 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:24.025158 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbsc6\" (UniqueName: \"kubernetes.io/projected/7a16f5b6-099a-4204-8c82-d28b8a48732c-kube-api-access-mbsc6\") pod \"maas-controller-77b55b5c58-wt88z\" (UID: \"7a16f5b6-099a-4204-8c82-d28b8a48732c\") " pod="opendatahub/maas-controller-77b55b5c58-wt88z" Jul 07 06:44:24.027393 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:24.027356 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7a16f5b6-099a-4204-8c82-d28b8a48732c-webhook-cert\") pod \"maas-controller-77b55b5c58-wt88z\" (UID: \"7a16f5b6-099a-4204-8c82-d28b8a48732c\") " pod="opendatahub/maas-controller-77b55b5c58-wt88z" Jul 07 06:44:24.034078 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:24.034058 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbsc6\" (UniqueName: \"kubernetes.io/projected/7a16f5b6-099a-4204-8c82-d28b8a48732c-kube-api-access-mbsc6\") pod \"maas-controller-77b55b5c58-wt88z\" (UID: \"7a16f5b6-099a-4204-8c82-d28b8a48732c\") " pod="opendatahub/maas-controller-77b55b5c58-wt88z" Jul 07 06:44:24.118052 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:24.118028 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77b55b5c58-wt88z" Jul 07 06:44:24.248160 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:24.248133 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77b55b5c58-wt88z"] Jul 07 06:44:24.250847 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:44:24.250824 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a16f5b6_099a_4204_8c82_d28b8a48732c.slice/crio-2f655a321bbc663b5a2c621729f7e8495d516490e53d5796b24e02795b7097b2 WatchSource:0}: Error finding container 2f655a321bbc663b5a2c621729f7e8495d516490e53d5796b24e02795b7097b2: Status 404 returned error can't find the container with id 2f655a321bbc663b5a2c621729f7e8495d516490e53d5796b24e02795b7097b2 Jul 07 06:44:24.252517 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:24.252502 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 06:44:24.302635 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:24.302603 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77b55b5c58-wt88z" event={"ID":"7a16f5b6-099a-4204-8c82-d28b8a48732c","Type":"ContainerStarted","Data":"2f655a321bbc663b5a2c621729f7e8495d516490e53d5796b24e02795b7097b2"} Jul 07 06:44:25.309468 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:25.309425 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77b55b5c58-wt88z" event={"ID":"7a16f5b6-099a-4204-8c82-d28b8a48732c","Type":"ContainerStarted","Data":"7e005e6c8d0d62b0c07255c75653d92e17bee6d56227c0d084c0a4dd8af55107"} Jul 07 06:44:25.309843 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:25.309566 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77b55b5c58-wt88z" Jul 07 06:44:25.326807 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:25.326760 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77b55b5c58-wt88z" podStartSLOduration=1.969047878 podStartE2EDuration="2.326742977s" totalImagesPullingTime="357.695099ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:44:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:44:25.326419312 +0000 UTC m=+741.166260281" watchObservedRunningTime="2026-07-07 06:44:25.326742977 +0000 UTC m=+741.166583947" Jul 07 06:44:36.321011 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:44:36.320982 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77b55b5c58-wt88z" Jul 07 06:45:02.711452 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:45:02.711413 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:45:02.811932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:45:02.811895 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:45:02.912424 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:45:02.912389 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:45:08.781466 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:45:08.781395 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:45:08.886642 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:45:08.886611 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:45:08.997732 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:45:08.997698 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:45:09.156079 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:45:09.156002 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:45:14.950734 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:45:14.950679 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:45:15.019734 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:45:15.019701 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:45:36.642869 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:45:36.642769 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:45:36.697344 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:45:36.697282 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:45:39.339272 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:45:39.339243 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:45:39.399968 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:45:39.399938 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:47:00.330527 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:47:00.330446 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:47:02.248559 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:47:02.248526 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:47:04.693620 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:47:04.693580 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 06:47:04.695574 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:47:04.695551 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 06:48:00.033777 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:48:00.033742 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:49:43.756074 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:49:43.756042 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:49:43.821675 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:49:43.821639 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:50:14.973256 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:50:14.973132 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:52:04.730886 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:52:04.730857 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 06:52:04.739426 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:52:04.739401 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 06:53:42.031820 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:53:42.031788 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:54:34.229309 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:54:34.229226 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:05.086659 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:05.086571 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:12.519338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:12.519291 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:20.801743 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:20.801703 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:26.739442 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:26.739402 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:28.732906 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:28.732870 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:28.825214 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:28.825172 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:29.187125 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:29.187094 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:35.238153 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.238119 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:35.342176 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.342136 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:35.422955 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.422930 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56"] Jul 07 06:56:35.426215 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.426199 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.429317 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.429301 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8e0f1335-openshift-default-dockercfg-kfpf7\"" Jul 07 06:56:35.455452 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.455400 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:35.455587 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.455523 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56"] Jul 07 06:56:35.460415 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.460392 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-workload-socket\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.460544 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.460421 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-data\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.460544 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.460442 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30541834-94f5-42eb-9b7a-444a6f9eda2a-istiod-ca-cert\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.460544 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.460475 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-envoy\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.460661 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.460539 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-podinfo\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.460661 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.460656 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-credential-socket\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.460724 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.460674 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-workload-certs\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.460724 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.460688 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-token\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.460724 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.460708 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szg95\" (UniqueName: \"kubernetes.io/projected/30541834-94f5-42eb-9b7a-444a6f9eda2a-kube-api-access-szg95\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.551512 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.551477 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:35.561436 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.561413 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30541834-94f5-42eb-9b7a-444a6f9eda2a-istiod-ca-cert\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.561573 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.561459 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-envoy\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.561573 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.561488 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-podinfo\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.561686 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.561607 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-credential-socket\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.561686 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.561634 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-workload-certs\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.561686 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.561658 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-token\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.561835 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.561687 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szg95\" (UniqueName: \"kubernetes.io/projected/30541834-94f5-42eb-9b7a-444a6f9eda2a-kube-api-access-szg95\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.561937 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.561906 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-credential-socket\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.562104 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.561956 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-workload-socket\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.562104 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.561991 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-data\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.562104 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.562035 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-workload-certs\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.562252 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.562209 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30541834-94f5-42eb-9b7a-444a6f9eda2a-istiod-ca-cert\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.562252 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.562242 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-workload-socket\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.562322 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.562263 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-data\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.564201 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.564183 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-envoy\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.564421 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.564406 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-podinfo\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.570655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.570635 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-token\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.571354 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.571330 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szg95\" (UniqueName: \"kubernetes.io/projected/30541834-94f5-42eb-9b7a-444a6f9eda2a-kube-api-access-szg95\") pod \"e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.736447 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.736357 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:35.894480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.894458 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56"] Jul 07 06:56:35.895979 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:56:35.895956 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30541834_94f5_42eb_9b7a_444a6f9eda2a.slice/crio-0ad20f0cbf10f487974fb8877e7b67792bcb3abc6cd03e0b03d27d649435dd7e WatchSource:0}: Error finding container 0ad20f0cbf10f487974fb8877e7b67792bcb3abc6cd03e0b03d27d649435dd7e: Status 404 returned error can't find the container with id 0ad20f0cbf10f487974fb8877e7b67792bcb3abc6cd03e0b03d27d649435dd7e Jul 07 06:56:35.901403 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.898706 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:56:35.901403 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.898879 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:56:35.901403 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:35.898986 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:56:36.020979 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.020940 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" event={"ID":"30541834-94f5-42eb-9b7a-444a6f9eda2a","Type":"ContainerStarted","Data":"b96067bc57464cd56c4934e32ae64cf8b3bb5f2675c1c34a73983fe73164d485"} Jul 07 06:56:36.021131 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.021024 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" event={"ID":"30541834-94f5-42eb-9b7a-444a6f9eda2a","Type":"ContainerStarted","Data":"0ad20f0cbf10f487974fb8877e7b67792bcb3abc6cd03e0b03d27d649435dd7e"} Jul 07 06:56:36.063613 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.063559 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" podStartSLOduration=1.063543243 podStartE2EDuration="1.063543243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:56:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:56:36.0608957 +0000 UTC m=+1471.900736668" watchObservedRunningTime="2026-07-07 06:56:36.063543243 +0000 UTC m=+1471.903384270" Jul 07 06:56:36.661602 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.661567 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:36.722825 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.722792 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm"] Jul 07 06:56:36.726313 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.726287 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:36.730139 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.730113 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-8e0f1335-serving-cert\"" Jul 07 06:56:36.730272 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.730210 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-754v8\"" Jul 07 06:56:36.736649 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.736630 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:36.741768 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.741748 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:36.743950 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.743929 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm"] Jul 07 06:56:36.775398 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.775356 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6cb271a-16a7-4d99-b71f-aeba64026578-maas-api-tls\") pod \"maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm\" (UID: \"b6cb271a-16a7-4d99-b71f-aeba64026578\") " pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:36.775489 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.775402 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6cb271a-16a7-4d99-b71f-aeba64026578-openshift-service-ca\") pod \"maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm\" (UID: \"b6cb271a-16a7-4d99-b71f-aeba64026578\") " pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:36.775489 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.775447 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz27s\" (UniqueName: \"kubernetes.io/projected/b6cb271a-16a7-4d99-b71f-aeba64026578-kube-api-access-rz27s\") pod \"maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm\" (UID: \"b6cb271a-16a7-4d99-b71f-aeba64026578\") " pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:36.783978 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.783956 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:36.876089 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.876054 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6cb271a-16a7-4d99-b71f-aeba64026578-maas-api-tls\") pod \"maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm\" (UID: \"b6cb271a-16a7-4d99-b71f-aeba64026578\") " pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:36.876089 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.876092 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6cb271a-16a7-4d99-b71f-aeba64026578-openshift-service-ca\") pod \"maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm\" (UID: \"b6cb271a-16a7-4d99-b71f-aeba64026578\") " pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:36.876310 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.876134 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz27s\" (UniqueName: \"kubernetes.io/projected/b6cb271a-16a7-4d99-b71f-aeba64026578-kube-api-access-rz27s\") pod \"maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm\" (UID: \"b6cb271a-16a7-4d99-b71f-aeba64026578\") " pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:36.876310 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:56:36.876202 2579 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-8e0f1335-serving-cert: secret "maas-api-e2e-ait-8e0f1335-serving-cert" not found Jul 07 06:56:36.876310 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:56:36.876264 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6cb271a-16a7-4d99-b71f-aeba64026578-maas-api-tls podName:b6cb271a-16a7-4d99-b71f-aeba64026578 nodeName:}" failed. No retries permitted until 2026-07-07 06:56:37.376247231 +0000 UTC m=+1473.216088176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b6cb271a-16a7-4d99-b71f-aeba64026578-maas-api-tls") pod "maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" (UID: "b6cb271a-16a7-4d99-b71f-aeba64026578") : secret "maas-api-e2e-ait-8e0f1335-serving-cert" not found Jul 07 06:56:36.876847 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.876825 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6cb271a-16a7-4d99-b71f-aeba64026578-openshift-service-ca\") pod \"maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm\" (UID: \"b6cb271a-16a7-4d99-b71f-aeba64026578\") " pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:36.894169 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:36.894139 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz27s\" (UniqueName: \"kubernetes.io/projected/b6cb271a-16a7-4d99-b71f-aeba64026578-kube-api-access-rz27s\") pod \"maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm\" (UID: \"b6cb271a-16a7-4d99-b71f-aeba64026578\") " pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:37.026345 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:37.026320 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:37.027237 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:37.027211 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:37.293486 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:37.293390 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:37.381517 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:37.381487 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6cb271a-16a7-4d99-b71f-aeba64026578-maas-api-tls\") pod \"maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm\" (UID: \"b6cb271a-16a7-4d99-b71f-aeba64026578\") " pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:37.381666 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:56:37.381602 2579 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-8e0f1335-serving-cert: secret "maas-api-e2e-ait-8e0f1335-serving-cert" not found Jul 07 06:56:37.381666 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:56:37.381654 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6cb271a-16a7-4d99-b71f-aeba64026578-maas-api-tls podName:b6cb271a-16a7-4d99-b71f-aeba64026578 nodeName:}" failed. No retries permitted until 2026-07-07 06:56:38.381640173 +0000 UTC m=+1474.221481118 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b6cb271a-16a7-4d99-b71f-aeba64026578-maas-api-tls") pod "maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" (UID: "b6cb271a-16a7-4d99-b71f-aeba64026578") : secret "maas-api-e2e-ait-8e0f1335-serving-cert" not found Jul 07 06:56:37.398538 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:37.398505 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:37.507705 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:37.507675 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:38.390797 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:38.390757 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6cb271a-16a7-4d99-b71f-aeba64026578-maas-api-tls\") pod \"maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm\" (UID: \"b6cb271a-16a7-4d99-b71f-aeba64026578\") " pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:38.393166 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:38.393144 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6cb271a-16a7-4d99-b71f-aeba64026578-maas-api-tls\") pod \"maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm\" (UID: \"b6cb271a-16a7-4d99-b71f-aeba64026578\") " pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:38.537006 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:38.536972 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:38.672689 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:38.672664 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm"] Jul 07 06:56:38.674266 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:56:38.674238 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6cb271a_16a7_4d99_b71f_aeba64026578.slice/crio-97058bdf0d7795a9968874f656b3662a456691c1954ddfaae8af87052854cab2 WatchSource:0}: Error finding container 97058bdf0d7795a9968874f656b3662a456691c1954ddfaae8af87052854cab2: Status 404 returned error can't find the container with id 97058bdf0d7795a9968874f656b3662a456691c1954ddfaae8af87052854cab2 Jul 07 06:56:38.675752 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:38.675733 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 06:56:38.895975 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:38.895915 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:39.037195 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:39.037159 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" event={"ID":"b6cb271a-16a7-4d99-b71f-aeba64026578","Type":"ContainerStarted","Data":"97058bdf0d7795a9968874f656b3662a456691c1954ddfaae8af87052854cab2"} Jul 07 06:56:40.043848 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:40.043817 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" event={"ID":"b6cb271a-16a7-4d99-b71f-aeba64026578","Type":"ContainerStarted","Data":"02ad108c1f979fafb5546824a042cc72bf9bd3711814f22432756731ba04db66"} Jul 07 06:56:40.044296 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:40.043890 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:40.063992 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:40.063945 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" podStartSLOduration=3.592137131 podStartE2EDuration="4.063930499s" totalImagesPullingTime="471.793368ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:56:40.062599159 +0000 UTC m=+1475.902440127" watchObservedRunningTime="2026-07-07 06:56:40.063930499 +0000 UTC m=+1475.903771468" Jul 07 06:56:41.664627 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:41.664582 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:41.835219 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:41.835176 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56"] Jul 07 06:56:41.835486 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:41.835454 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" podUID="30541834-94f5-42eb-9b7a-444a6f9eda2a" containerName="istio-proxy" containerID="cri-o://b96067bc57464cd56c4934e32ae64cf8b3bb5f2675c1c34a73983fe73164d485" gracePeriod=30 Jul 07 06:56:41.854197 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:41.854161 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:41.958892 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:41.958807 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:42.013307 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:42.013270 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm"] Jul 07 06:56:42.059378 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:42.059325 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" podUID="b6cb271a-16a7-4d99-b71f-aeba64026578" containerName="maas-api" containerID="cri-o://02ad108c1f979fafb5546824a042cc72bf9bd3711814f22432756731ba04db66" gracePeriod=30 Jul 07 06:56:42.059653 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:42.059508 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:42.348573 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:42.348542 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:42.453733 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:42.453695 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rz27s\" (UniqueName: \"kubernetes.io/projected/b6cb271a-16a7-4d99-b71f-aeba64026578-kube-api-access-rz27s\") pod \"b6cb271a-16a7-4d99-b71f-aeba64026578\" (UID: \"b6cb271a-16a7-4d99-b71f-aeba64026578\") " Jul 07 06:56:42.453918 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:42.453805 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6cb271a-16a7-4d99-b71f-aeba64026578-openshift-service-ca\") pod \"b6cb271a-16a7-4d99-b71f-aeba64026578\" (UID: \"b6cb271a-16a7-4d99-b71f-aeba64026578\") " Jul 07 06:56:42.453918 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:42.453855 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6cb271a-16a7-4d99-b71f-aeba64026578-maas-api-tls\") pod \"b6cb271a-16a7-4d99-b71f-aeba64026578\" (UID: \"b6cb271a-16a7-4d99-b71f-aeba64026578\") " Jul 07 06:56:42.454198 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:42.454163 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6cb271a-16a7-4d99-b71f-aeba64026578-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b6cb271a-16a7-4d99-b71f-aeba64026578" (UID: "b6cb271a-16a7-4d99-b71f-aeba64026578"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 06:56:42.455833 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:42.455808 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6cb271a-16a7-4d99-b71f-aeba64026578-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b6cb271a-16a7-4d99-b71f-aeba64026578" (UID: "b6cb271a-16a7-4d99-b71f-aeba64026578"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 06:56:42.455996 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:42.455967 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6cb271a-16a7-4d99-b71f-aeba64026578-kube-api-access-rz27s" (OuterVolumeSpecName: "kube-api-access-rz27s") pod "b6cb271a-16a7-4d99-b71f-aeba64026578" (UID: "b6cb271a-16a7-4d99-b71f-aeba64026578"). InnerVolumeSpecName "kube-api-access-rz27s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:56:42.554854 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:42.554812 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rz27s\" (UniqueName: \"kubernetes.io/projected/b6cb271a-16a7-4d99-b71f-aeba64026578-kube-api-access-rz27s\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:56:42.554854 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:42.554850 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6cb271a-16a7-4d99-b71f-aeba64026578-openshift-service-ca\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:56:42.555033 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:42.554866 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6cb271a-16a7-4d99-b71f-aeba64026578-maas-api-tls\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:56:43.066899 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:43.066862 2579 generic.go:356] "Generic (PLEG): container finished" podID="b6cb271a-16a7-4d99-b71f-aeba64026578" containerID="02ad108c1f979fafb5546824a042cc72bf9bd3711814f22432756731ba04db66" exitCode=0 Jul 07 06:56:43.067325 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:43.066910 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" Jul 07 06:56:43.067325 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:43.066944 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" event={"ID":"b6cb271a-16a7-4d99-b71f-aeba64026578","Type":"ContainerDied","Data":"02ad108c1f979fafb5546824a042cc72bf9bd3711814f22432756731ba04db66"} Jul 07 06:56:43.067325 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:43.067001 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm" event={"ID":"b6cb271a-16a7-4d99-b71f-aeba64026578","Type":"ContainerDied","Data":"97058bdf0d7795a9968874f656b3662a456691c1954ddfaae8af87052854cab2"} Jul 07 06:56:43.067325 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:43.067025 2579 scope.go:117] "RemoveContainer" containerID="02ad108c1f979fafb5546824a042cc72bf9bd3711814f22432756731ba04db66" Jul 07 06:56:43.075961 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:43.075933 2579 scope.go:117] "RemoveContainer" containerID="02ad108c1f979fafb5546824a042cc72bf9bd3711814f22432756731ba04db66" Jul 07 06:56:43.076210 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:56:43.076194 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02ad108c1f979fafb5546824a042cc72bf9bd3711814f22432756731ba04db66\": container with ID starting with 02ad108c1f979fafb5546824a042cc72bf9bd3711814f22432756731ba04db66 not found: ID does not exist" containerID="02ad108c1f979fafb5546824a042cc72bf9bd3711814f22432756731ba04db66" Jul 07 06:56:43.076271 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:43.076218 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02ad108c1f979fafb5546824a042cc72bf9bd3711814f22432756731ba04db66"} err="failed to get container status \"02ad108c1f979fafb5546824a042cc72bf9bd3711814f22432756731ba04db66\": rpc error: code = NotFound desc = could not find container \"02ad108c1f979fafb5546824a042cc72bf9bd3711814f22432756731ba04db66\": container with ID starting with 02ad108c1f979fafb5546824a042cc72bf9bd3711814f22432756731ba04db66 not found: ID does not exist" Jul 07 06:56:43.085331 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:43.085310 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm"] Jul 07 06:56:43.089621 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:43.089601 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-8e0f1335-76d66ff7cc-x89qm"] Jul 07 06:56:44.785181 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:44.785146 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6cb271a-16a7-4d99-b71f-aeba64026578" path="/var/lib/kubelet/pods/b6cb271a-16a7-4d99-b71f-aeba64026578/volumes" Jul 07 06:56:45.332707 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:45.332663 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:47.485748 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.485694 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:47.612664 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.612581 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30541834-94f5-42eb-9b7a-444a6f9eda2a-istiod-ca-cert\") pod \"30541834-94f5-42eb-9b7a-444a6f9eda2a\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " Jul 07 06:56:47.612664 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.612634 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szg95\" (UniqueName: \"kubernetes.io/projected/30541834-94f5-42eb-9b7a-444a6f9eda2a-kube-api-access-szg95\") pod \"30541834-94f5-42eb-9b7a-444a6f9eda2a\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " Jul 07 06:56:47.612664 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.612662 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-credential-socket\") pod \"30541834-94f5-42eb-9b7a-444a6f9eda2a\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " Jul 07 06:56:47.612937 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.612697 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-token\") pod \"30541834-94f5-42eb-9b7a-444a6f9eda2a\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " Jul 07 06:56:47.612937 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.612745 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-workload-socket\") pod \"30541834-94f5-42eb-9b7a-444a6f9eda2a\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " Jul 07 06:56:47.612937 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.612887 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-envoy\") pod \"30541834-94f5-42eb-9b7a-444a6f9eda2a\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " Jul 07 06:56:47.613148 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.612966 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-podinfo\") pod \"30541834-94f5-42eb-9b7a-444a6f9eda2a\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " Jul 07 06:56:47.613148 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.612997 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-workload-certs\") pod \"30541834-94f5-42eb-9b7a-444a6f9eda2a\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " Jul 07 06:56:47.613148 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.612997 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "30541834-94f5-42eb-9b7a-444a6f9eda2a" (UID: "30541834-94f5-42eb-9b7a-444a6f9eda2a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:56:47.613148 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.613004 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "30541834-94f5-42eb-9b7a-444a6f9eda2a" (UID: "30541834-94f5-42eb-9b7a-444a6f9eda2a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:56:47.613148 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.613026 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-data\") pod \"30541834-94f5-42eb-9b7a-444a6f9eda2a\" (UID: \"30541834-94f5-42eb-9b7a-444a6f9eda2a\") " Jul 07 06:56:47.613148 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.613053 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30541834-94f5-42eb-9b7a-444a6f9eda2a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "30541834-94f5-42eb-9b7a-444a6f9eda2a" (UID: "30541834-94f5-42eb-9b7a-444a6f9eda2a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 06:56:47.613507 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.613243 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "30541834-94f5-42eb-9b7a-444a6f9eda2a" (UID: "30541834-94f5-42eb-9b7a-444a6f9eda2a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:56:47.613507 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.613418 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30541834-94f5-42eb-9b7a-444a6f9eda2a-istiod-ca-cert\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:56:47.613507 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.613435 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-credential-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:56:47.613507 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.613448 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-workload-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:56:47.613507 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.613464 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-workload-certs\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:56:47.613761 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.613723 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-data" (OuterVolumeSpecName: "istio-data") pod "30541834-94f5-42eb-9b7a-444a6f9eda2a" (UID: "30541834-94f5-42eb-9b7a-444a6f9eda2a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:56:47.614963 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.614942 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-token" (OuterVolumeSpecName: "istio-token") pod "30541834-94f5-42eb-9b7a-444a6f9eda2a" (UID: "30541834-94f5-42eb-9b7a-444a6f9eda2a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:56:47.615059 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.615008 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "30541834-94f5-42eb-9b7a-444a6f9eda2a" (UID: "30541834-94f5-42eb-9b7a-444a6f9eda2a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 06:56:47.615153 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.615137 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "30541834-94f5-42eb-9b7a-444a6f9eda2a" (UID: "30541834-94f5-42eb-9b7a-444a6f9eda2a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:56:47.615213 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.615167 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30541834-94f5-42eb-9b7a-444a6f9eda2a-kube-api-access-szg95" (OuterVolumeSpecName: "kube-api-access-szg95") pod "30541834-94f5-42eb-9b7a-444a6f9eda2a" (UID: "30541834-94f5-42eb-9b7a-444a6f9eda2a"). InnerVolumeSpecName "kube-api-access-szg95". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:56:47.714746 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.714716 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-podinfo\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:56:47.714746 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.714742 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-data\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:56:47.714909 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.714755 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szg95\" (UniqueName: \"kubernetes.io/projected/30541834-94f5-42eb-9b7a-444a6f9eda2a-kube-api-access-szg95\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:56:47.714909 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.714768 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-token\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:56:47.714909 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.714779 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30541834-94f5-42eb-9b7a-444a6f9eda2a-istio-envoy\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:56:47.992872 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:47.992787 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:48.087744 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.087700 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:48.096860 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.096762 2579 generic.go:356] "Generic (PLEG): container finished" podID="30541834-94f5-42eb-9b7a-444a6f9eda2a" containerID="b96067bc57464cd56c4934e32ae64cf8b3bb5f2675c1c34a73983fe73164d485" exitCode=0 Jul 07 06:56:48.096860 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.096819 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" Jul 07 06:56:48.097095 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.096863 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" event={"ID":"30541834-94f5-42eb-9b7a-444a6f9eda2a","Type":"ContainerDied","Data":"b96067bc57464cd56c4934e32ae64cf8b3bb5f2675c1c34a73983fe73164d485"} Jul 07 06:56:48.097095 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.096918 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56" event={"ID":"30541834-94f5-42eb-9b7a-444a6f9eda2a","Type":"ContainerDied","Data":"0ad20f0cbf10f487974fb8877e7b67792bcb3abc6cd03e0b03d27d649435dd7e"} Jul 07 06:56:48.097095 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.096945 2579 scope.go:117] "RemoveContainer" containerID="b96067bc57464cd56c4934e32ae64cf8b3bb5f2675c1c34a73983fe73164d485" Jul 07 06:56:48.106928 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.106910 2579 scope.go:117] "RemoveContainer" containerID="b96067bc57464cd56c4934e32ae64cf8b3bb5f2675c1c34a73983fe73164d485" Jul 07 06:56:48.107212 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:56:48.107187 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b96067bc57464cd56c4934e32ae64cf8b3bb5f2675c1c34a73983fe73164d485\": container with ID starting with b96067bc57464cd56c4934e32ae64cf8b3bb5f2675c1c34a73983fe73164d485 not found: ID does not exist" containerID="b96067bc57464cd56c4934e32ae64cf8b3bb5f2675c1c34a73983fe73164d485" Jul 07 06:56:48.107344 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.107215 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b96067bc57464cd56c4934e32ae64cf8b3bb5f2675c1c34a73983fe73164d485"} err="failed to get container status \"b96067bc57464cd56c4934e32ae64cf8b3bb5f2675c1c34a73983fe73164d485\": rpc error: code = NotFound desc = could not find container \"b96067bc57464cd56c4934e32ae64cf8b3bb5f2675c1c34a73983fe73164d485\": container with ID starting with b96067bc57464cd56c4934e32ae64cf8b3bb5f2675c1c34a73983fe73164d485 not found: ID does not exist" Jul 07 06:56:48.129088 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.129064 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56"] Jul 07 06:56:48.143601 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.143574 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8e0f1335-openshift-default-7cbcfbf55c-bzn56"] Jul 07 06:56:48.184427 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.184395 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f"] Jul 07 06:56:48.184988 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.184962 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30541834-94f5-42eb-9b7a-444a6f9eda2a" containerName="istio-proxy" Jul 07 06:56:48.185106 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.185012 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="30541834-94f5-42eb-9b7a-444a6f9eda2a" containerName="istio-proxy" Jul 07 06:56:48.185106 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.185025 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6cb271a-16a7-4d99-b71f-aeba64026578" containerName="maas-api" Jul 07 06:56:48.185106 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.185036 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6cb271a-16a7-4d99-b71f-aeba64026578" containerName="maas-api" Jul 07 06:56:48.185279 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.185245 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="30541834-94f5-42eb-9b7a-444a6f9eda2a" containerName="istio-proxy" Jul 07 06:56:48.185279 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.185264 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6cb271a-16a7-4d99-b71f-aeba64026578" containerName="maas-api" Jul 07 06:56:48.188108 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.188090 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.190759 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.190738 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5eda7da0-openshift-default-dockercfg-krz8p\"" Jul 07 06:56:48.205489 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.205465 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f"] Jul 07 06:56:48.250783 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.250715 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:48.319905 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.319874 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-token\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.320065 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.319924 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-credential-socket\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.320065 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.319948 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istiod-ca-cert\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.320065 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.320031 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-data\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.320164 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.320076 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-podinfo\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.320164 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.320117 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-workload-certs\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.320229 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.320185 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpdd4\" (UniqueName: \"kubernetes.io/projected/709e6f5c-7595-40c3-a223-6b86e4eec5cc-kube-api-access-qpdd4\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.320229 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.320208 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-workload-socket\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.320229 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.320224 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-envoy\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.400428 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.400392 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:48.420921 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.420892 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpdd4\" (UniqueName: \"kubernetes.io/projected/709e6f5c-7595-40c3-a223-6b86e4eec5cc-kube-api-access-qpdd4\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.421054 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.420924 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-workload-socket\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.421054 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.420945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-envoy\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.421054 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.420968 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-token\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.421054 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.420997 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-credential-socket\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.421054 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.421016 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istiod-ca-cert\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.421054 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.421051 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-data\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.421579 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.421142 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-podinfo\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.421579 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.421184 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-workload-certs\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.421579 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.421343 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-workload-socket\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.421579 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.421523 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-credential-socket\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.421579 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.421535 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-workload-certs\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.421891 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.421589 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-data\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.421891 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.421697 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istiod-ca-cert\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.423432 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.423405 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-envoy\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.423648 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.423627 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-podinfo\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.432894 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.432869 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpdd4\" (UniqueName: \"kubernetes.io/projected/709e6f5c-7595-40c3-a223-6b86e4eec5cc-kube-api-access-qpdd4\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.436263 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.436233 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-token\") pod \"e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.500472 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.500444 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:48.637453 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.637426 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f"] Jul 07 06:56:48.639599 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:56:48.639558 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod709e6f5c_7595_40c3_a223_6b86e4eec5cc.slice/crio-b1180210a52f0ec30ebbd5bf316486abbd7153b2a5e57c1d41e15a89409cf61f WatchSource:0}: Error finding container b1180210a52f0ec30ebbd5bf316486abbd7153b2a5e57c1d41e15a89409cf61f: Status 404 returned error can't find the container with id b1180210a52f0ec30ebbd5bf316486abbd7153b2a5e57c1d41e15a89409cf61f Jul 07 06:56:48.642789 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.642757 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:56:48.642906 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.642833 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:56:48.642906 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.642881 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:56:48.783552 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:48.783522 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30541834-94f5-42eb-9b7a-444a6f9eda2a" path="/var/lib/kubelet/pods/30541834-94f5-42eb-9b7a-444a6f9eda2a/volumes" Jul 07 06:56:49.105651 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.105556 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" event={"ID":"709e6f5c-7595-40c3-a223-6b86e4eec5cc","Type":"ContainerStarted","Data":"d1e811369aa4b24913ec566f3cb48bad2cc445d2bc855041ba4e6ed4d09887d0"} Jul 07 06:56:49.105651 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.105617 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" event={"ID":"709e6f5c-7595-40c3-a223-6b86e4eec5cc","Type":"ContainerStarted","Data":"b1180210a52f0ec30ebbd5bf316486abbd7153b2a5e57c1d41e15a89409cf61f"} Jul 07 06:56:49.133590 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.133536 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" podStartSLOduration=1.133515915 podStartE2EDuration="1.133515915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:56:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:56:49.132195091 +0000 UTC m=+1484.972036060" watchObservedRunningTime="2026-07-07 06:56:49.133515915 +0000 UTC m=+1484.973356886" Jul 07 06:56:49.403881 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.403800 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:49.490669 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.490635 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:49.500911 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.500883 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:49.505636 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.505617 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:49.605657 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.605619 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q"] Jul 07 06:56:49.608296 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.608272 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" Jul 07 06:56:49.611833 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.611810 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-754v8\"" Jul 07 06:56:49.612151 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.612135 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-5eda7da0-serving-cert\"" Jul 07 06:56:49.627727 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.627701 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q"] Jul 07 06:56:49.661751 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.661696 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:49.733684 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.733647 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6dk5\" (UniqueName: \"kubernetes.io/projected/30600b8f-7944-45ff-9a0a-51a70f926659-kube-api-access-h6dk5\") pod \"maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q\" (UID: \"30600b8f-7944-45ff-9a0a-51a70f926659\") " pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" Jul 07 06:56:49.733843 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.733742 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30600b8f-7944-45ff-9a0a-51a70f926659-maas-api-tls\") pod \"maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q\" (UID: \"30600b8f-7944-45ff-9a0a-51a70f926659\") " pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" Jul 07 06:56:49.733843 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.733802 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30600b8f-7944-45ff-9a0a-51a70f926659-openshift-service-ca\") pod \"maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q\" (UID: \"30600b8f-7944-45ff-9a0a-51a70f926659\") " pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" Jul 07 06:56:49.809682 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.809648 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:49.834819 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.834795 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6dk5\" (UniqueName: \"kubernetes.io/projected/30600b8f-7944-45ff-9a0a-51a70f926659-kube-api-access-h6dk5\") pod \"maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q\" (UID: \"30600b8f-7944-45ff-9a0a-51a70f926659\") " pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" Jul 07 06:56:49.834925 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.834867 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30600b8f-7944-45ff-9a0a-51a70f926659-maas-api-tls\") pod \"maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q\" (UID: \"30600b8f-7944-45ff-9a0a-51a70f926659\") " pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" Jul 07 06:56:49.834992 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.834946 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30600b8f-7944-45ff-9a0a-51a70f926659-openshift-service-ca\") pod \"maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q\" (UID: \"30600b8f-7944-45ff-9a0a-51a70f926659\") " pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" Jul 07 06:56:49.835560 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.835541 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30600b8f-7944-45ff-9a0a-51a70f926659-openshift-service-ca\") pod \"maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q\" (UID: \"30600b8f-7944-45ff-9a0a-51a70f926659\") " pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" Jul 07 06:56:49.837207 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.837180 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30600b8f-7944-45ff-9a0a-51a70f926659-maas-api-tls\") pod \"maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q\" (UID: \"30600b8f-7944-45ff-9a0a-51a70f926659\") " pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" Jul 07 06:56:49.845899 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.845877 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6dk5\" (UniqueName: \"kubernetes.io/projected/30600b8f-7944-45ff-9a0a-51a70f926659-kube-api-access-h6dk5\") pod \"maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q\" (UID: \"30600b8f-7944-45ff-9a0a-51a70f926659\") " pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" Jul 07 06:56:49.918933 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:49.918870 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" Jul 07 06:56:50.056217 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:50.056184 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q"] Jul 07 06:56:50.057445 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:56:50.057418 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30600b8f_7944_45ff_9a0a_51a70f926659.slice/crio-222db929aaccdfe0172e52745d10fc11d419be123a460102fca389a78ee1354c WatchSource:0}: Error finding container 222db929aaccdfe0172e52745d10fc11d419be123a460102fca389a78ee1354c: Status 404 returned error can't find the container with id 222db929aaccdfe0172e52745d10fc11d419be123a460102fca389a78ee1354c Jul 07 06:56:50.111723 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:50.111680 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" event={"ID":"30600b8f-7944-45ff-9a0a-51a70f926659","Type":"ContainerStarted","Data":"222db929aaccdfe0172e52745d10fc11d419be123a460102fca389a78ee1354c"} Jul 07 06:56:50.111944 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:50.111923 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:50.112736 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:50.112718 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:56:50.270975 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:50.270939 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:50.377461 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:50.377433 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:51.119697 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:51.119661 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" event={"ID":"30600b8f-7944-45ff-9a0a-51a70f926659","Type":"ContainerStarted","Data":"9ddd0ff2772270f4f7b0191e73635a439065ccc26316440c3ac3c419be8e85d8"} Jul 07 06:56:51.120113 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:51.119814 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" Jul 07 06:56:51.141055 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:51.141002 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" podStartSLOduration=1.833132155 podStartE2EDuration="2.140987595s" totalImagesPullingTime="307.85544ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:56:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:56:51.139807411 +0000 UTC m=+1486.979648392" watchObservedRunningTime="2026-07-07 06:56:51.140987595 +0000 UTC m=+1486.980828562" Jul 07 06:56:51.449118 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:51.449084 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:54.438630 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:54.438594 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:54.739119 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:54.739029 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q"] Jul 07 06:56:54.739284 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:54.739235 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" podUID="30600b8f-7944-45ff-9a0a-51a70f926659" containerName="maas-api" containerID="cri-o://9ddd0ff2772270f4f7b0191e73635a439065ccc26316440c3ac3c419be8e85d8" gracePeriod=30 Jul 07 06:56:54.770724 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:54.770690 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:56:54.986342 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:54.986320 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" Jul 07 06:56:55.084556 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.084528 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6dk5\" (UniqueName: \"kubernetes.io/projected/30600b8f-7944-45ff-9a0a-51a70f926659-kube-api-access-h6dk5\") pod \"30600b8f-7944-45ff-9a0a-51a70f926659\" (UID: \"30600b8f-7944-45ff-9a0a-51a70f926659\") " Jul 07 06:56:55.084734 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.084578 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30600b8f-7944-45ff-9a0a-51a70f926659-openshift-service-ca\") pod \"30600b8f-7944-45ff-9a0a-51a70f926659\" (UID: \"30600b8f-7944-45ff-9a0a-51a70f926659\") " Jul 07 06:56:55.084734 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.084634 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30600b8f-7944-45ff-9a0a-51a70f926659-maas-api-tls\") pod \"30600b8f-7944-45ff-9a0a-51a70f926659\" (UID: \"30600b8f-7944-45ff-9a0a-51a70f926659\") " Jul 07 06:56:55.085002 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.084978 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30600b8f-7944-45ff-9a0a-51a70f926659-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "30600b8f-7944-45ff-9a0a-51a70f926659" (UID: "30600b8f-7944-45ff-9a0a-51a70f926659"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 06:56:55.086657 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.086632 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30600b8f-7944-45ff-9a0a-51a70f926659-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "30600b8f-7944-45ff-9a0a-51a70f926659" (UID: "30600b8f-7944-45ff-9a0a-51a70f926659"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 06:56:55.086786 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.086771 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30600b8f-7944-45ff-9a0a-51a70f926659-kube-api-access-h6dk5" (OuterVolumeSpecName: "kube-api-access-h6dk5") pod "30600b8f-7944-45ff-9a0a-51a70f926659" (UID: "30600b8f-7944-45ff-9a0a-51a70f926659"). InnerVolumeSpecName "kube-api-access-h6dk5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:56:55.142350 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.142317 2579 generic.go:356] "Generic (PLEG): container finished" podID="30600b8f-7944-45ff-9a0a-51a70f926659" containerID="9ddd0ff2772270f4f7b0191e73635a439065ccc26316440c3ac3c419be8e85d8" exitCode=0 Jul 07 06:56:55.142503 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.142389 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" Jul 07 06:56:55.142503 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.142410 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" event={"ID":"30600b8f-7944-45ff-9a0a-51a70f926659","Type":"ContainerDied","Data":"9ddd0ff2772270f4f7b0191e73635a439065ccc26316440c3ac3c419be8e85d8"} Jul 07 06:56:55.142503 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.142501 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q" event={"ID":"30600b8f-7944-45ff-9a0a-51a70f926659","Type":"ContainerDied","Data":"222db929aaccdfe0172e52745d10fc11d419be123a460102fca389a78ee1354c"} Jul 07 06:56:55.142712 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.142532 2579 scope.go:117] "RemoveContainer" containerID="9ddd0ff2772270f4f7b0191e73635a439065ccc26316440c3ac3c419be8e85d8" Jul 07 06:56:55.151914 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.151897 2579 scope.go:117] "RemoveContainer" containerID="9ddd0ff2772270f4f7b0191e73635a439065ccc26316440c3ac3c419be8e85d8" Jul 07 06:56:55.152149 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:56:55.152129 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ddd0ff2772270f4f7b0191e73635a439065ccc26316440c3ac3c419be8e85d8\": container with ID starting with 9ddd0ff2772270f4f7b0191e73635a439065ccc26316440c3ac3c419be8e85d8 not found: ID does not exist" containerID="9ddd0ff2772270f4f7b0191e73635a439065ccc26316440c3ac3c419be8e85d8" Jul 07 06:56:55.152215 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.152160 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ddd0ff2772270f4f7b0191e73635a439065ccc26316440c3ac3c419be8e85d8"} err="failed to get container status \"9ddd0ff2772270f4f7b0191e73635a439065ccc26316440c3ac3c419be8e85d8\": rpc error: code = NotFound desc = could not find container \"9ddd0ff2772270f4f7b0191e73635a439065ccc26316440c3ac3c419be8e85d8\": container with ID starting with 9ddd0ff2772270f4f7b0191e73635a439065ccc26316440c3ac3c419be8e85d8 not found: ID does not exist" Jul 07 06:56:55.165793 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.165770 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q"] Jul 07 06:56:55.170291 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.170266 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-5eda7da0-cb87d6bdc-rq26q"] Jul 07 06:56:55.185320 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.185301 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6dk5\" (UniqueName: \"kubernetes.io/projected/30600b8f-7944-45ff-9a0a-51a70f926659-kube-api-access-h6dk5\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:56:55.185409 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.185321 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30600b8f-7944-45ff-9a0a-51a70f926659-openshift-service-ca\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:56:55.185409 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:55.185335 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30600b8f-7944-45ff-9a0a-51a70f926659-maas-api-tls\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:56:56.783151 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:56:56.783121 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30600b8f-7944-45ff-9a0a-51a70f926659" path="/var/lib/kubelet/pods/30600b8f-7944-45ff-9a0a-51a70f926659/volumes" Jul 07 06:57:00.555070 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:00.555038 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f"] Jul 07 06:57:00.555491 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:00.555281 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" podUID="709e6f5c-7595-40c3-a223-6b86e4eec5cc" containerName="istio-proxy" containerID="cri-o://d1e811369aa4b24913ec566f3cb48bad2cc445d2bc855041ba4e6ed4d09887d0" gracePeriod=30 Jul 07 06:57:00.595558 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:00.595516 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:00.677324 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:00.677286 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:02.750990 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:02.750948 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:04.771956 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:04.771931 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 06:57:04.775653 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:04.775633 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 06:57:05.114564 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.114520 2579 patch_prober.go:28] interesting pod/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 06:57:05.114712 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.114580 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" podUID="709e6f5c-7595-40c3-a223-6b86e4eec5cc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 06:57:05.803257 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.803234 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:57:05.873981 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.873899 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-podinfo\") pod \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " Jul 07 06:57:05.873981 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.873948 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-envoy\") pod \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " Jul 07 06:57:05.873981 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.873982 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-workload-certs\") pod \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " Jul 07 06:57:05.874240 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.873998 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-workload-socket\") pod \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " Jul 07 06:57:05.874240 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.874029 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-token\") pod \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " Jul 07 06:57:05.874240 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.874045 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istiod-ca-cert\") pod \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " Jul 07 06:57:05.874240 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.874072 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-credential-socket\") pod \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " Jul 07 06:57:05.874240 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.874106 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpdd4\" (UniqueName: \"kubernetes.io/projected/709e6f5c-7595-40c3-a223-6b86e4eec5cc-kube-api-access-qpdd4\") pod \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " Jul 07 06:57:05.874508 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.874356 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "709e6f5c-7595-40c3-a223-6b86e4eec5cc" (UID: "709e6f5c-7595-40c3-a223-6b86e4eec5cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:57:05.874508 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.874458 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-data\") pod \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\" (UID: \"709e6f5c-7595-40c3-a223-6b86e4eec5cc\") " Jul 07 06:57:05.874605 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.874516 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "709e6f5c-7595-40c3-a223-6b86e4eec5cc" (UID: "709e6f5c-7595-40c3-a223-6b86e4eec5cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:57:05.874605 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.874533 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "709e6f5c-7595-40c3-a223-6b86e4eec5cc" (UID: "709e6f5c-7595-40c3-a223-6b86e4eec5cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 06:57:05.874605 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.874587 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "709e6f5c-7595-40c3-a223-6b86e4eec5cc" (UID: "709e6f5c-7595-40c3-a223-6b86e4eec5cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:57:05.874805 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.874776 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-data" (OuterVolumeSpecName: "istio-data") pod "709e6f5c-7595-40c3-a223-6b86e4eec5cc" (UID: "709e6f5c-7595-40c3-a223-6b86e4eec5cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:57:05.875171 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.875002 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-workload-certs\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:05.875171 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.875030 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-workload-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:05.875171 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.875046 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istiod-ca-cert\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:05.875171 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.875060 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-credential-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:05.875171 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.875074 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-data\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:05.876094 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.876064 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "709e6f5c-7595-40c3-a223-6b86e4eec5cc" (UID: "709e6f5c-7595-40c3-a223-6b86e4eec5cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 06:57:05.876270 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.876245 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-token" (OuterVolumeSpecName: "istio-token") pod "709e6f5c-7595-40c3-a223-6b86e4eec5cc" (UID: "709e6f5c-7595-40c3-a223-6b86e4eec5cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:57:05.876605 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.876586 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "709e6f5c-7595-40c3-a223-6b86e4eec5cc" (UID: "709e6f5c-7595-40c3-a223-6b86e4eec5cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:57:05.876670 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.876601 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/709e6f5c-7595-40c3-a223-6b86e4eec5cc-kube-api-access-qpdd4" (OuterVolumeSpecName: "kube-api-access-qpdd4") pod "709e6f5c-7595-40c3-a223-6b86e4eec5cc" (UID: "709e6f5c-7595-40c3-a223-6b86e4eec5cc"). InnerVolumeSpecName "kube-api-access-qpdd4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:57:05.975934 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.975908 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-token\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:05.975934 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.975931 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpdd4\" (UniqueName: \"kubernetes.io/projected/709e6f5c-7595-40c3-a223-6b86e4eec5cc-kube-api-access-qpdd4\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:05.975934 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.975942 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-podinfo\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:05.976152 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:05.975950 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/709e6f5c-7595-40c3-a223-6b86e4eec5cc-istio-envoy\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:06.202795 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.202700 2579 generic.go:356] "Generic (PLEG): container finished" podID="709e6f5c-7595-40c3-a223-6b86e4eec5cc" containerID="d1e811369aa4b24913ec566f3cb48bad2cc445d2bc855041ba4e6ed4d09887d0" exitCode=0 Jul 07 06:57:06.202795 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.202751 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" Jul 07 06:57:06.202795 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.202774 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" event={"ID":"709e6f5c-7595-40c3-a223-6b86e4eec5cc","Type":"ContainerDied","Data":"d1e811369aa4b24913ec566f3cb48bad2cc445d2bc855041ba4e6ed4d09887d0"} Jul 07 06:57:06.203076 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.202822 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f" event={"ID":"709e6f5c-7595-40c3-a223-6b86e4eec5cc","Type":"ContainerDied","Data":"b1180210a52f0ec30ebbd5bf316486abbd7153b2a5e57c1d41e15a89409cf61f"} Jul 07 06:57:06.203076 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.202846 2579 scope.go:117] "RemoveContainer" containerID="d1e811369aa4b24913ec566f3cb48bad2cc445d2bc855041ba4e6ed4d09887d0" Jul 07 06:57:06.212089 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.212074 2579 scope.go:117] "RemoveContainer" containerID="d1e811369aa4b24913ec566f3cb48bad2cc445d2bc855041ba4e6ed4d09887d0" Jul 07 06:57:06.212305 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:57:06.212286 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1e811369aa4b24913ec566f3cb48bad2cc445d2bc855041ba4e6ed4d09887d0\": container with ID starting with d1e811369aa4b24913ec566f3cb48bad2cc445d2bc855041ba4e6ed4d09887d0 not found: ID does not exist" containerID="d1e811369aa4b24913ec566f3cb48bad2cc445d2bc855041ba4e6ed4d09887d0" Jul 07 06:57:06.212343 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.212316 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1e811369aa4b24913ec566f3cb48bad2cc445d2bc855041ba4e6ed4d09887d0"} err="failed to get container status \"d1e811369aa4b24913ec566f3cb48bad2cc445d2bc855041ba4e6ed4d09887d0\": rpc error: code = NotFound desc = could not find container \"d1e811369aa4b24913ec566f3cb48bad2cc445d2bc855041ba4e6ed4d09887d0\": container with ID starting with d1e811369aa4b24913ec566f3cb48bad2cc445d2bc855041ba4e6ed4d09887d0 not found: ID does not exist" Jul 07 06:57:06.226109 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.226087 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f"] Jul 07 06:57:06.230760 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.230739 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5eda7da0-openshift-default-77cf765-xgf8f"] Jul 07 06:57:06.694328 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.694279 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:06.782982 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.782947 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="709e6f5c-7595-40c3-a223-6b86e4eec5cc" path="/var/lib/kubelet/pods/709e6f5c-7595-40c3-a223-6b86e4eec5cc/volumes" Jul 07 06:57:06.822833 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.822790 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:06.966313 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.966234 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8"] Jul 07 06:57:06.966609 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.966596 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30600b8f-7944-45ff-9a0a-51a70f926659" containerName="maas-api" Jul 07 06:57:06.966654 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.966610 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="30600b8f-7944-45ff-9a0a-51a70f926659" containerName="maas-api" Jul 07 06:57:06.966654 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.966623 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="709e6f5c-7595-40c3-a223-6b86e4eec5cc" containerName="istio-proxy" Jul 07 06:57:06.966654 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.966628 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="709e6f5c-7595-40c3-a223-6b86e4eec5cc" containerName="istio-proxy" Jul 07 06:57:06.966750 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.966682 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="30600b8f-7944-45ff-9a0a-51a70f926659" containerName="maas-api" Jul 07 06:57:06.966750 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.966692 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="709e6f5c-7595-40c3-a223-6b86e4eec5cc" containerName="istio-proxy" Jul 07 06:57:06.969449 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.969433 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:06.973895 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.973878 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-2905275a-openshift-default-dockercfg-dq7lg\"" Jul 07 06:57:06.985691 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.985663 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-credential-socket\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:06.985806 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.985696 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-workload-certs\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:06.985806 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.985740 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-envoy\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:06.985806 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.985763 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-podinfo\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:06.985806 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.985785 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-token\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:06.985967 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.985887 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19c025e8-48e4-4a4c-9411-cd009a7e321c-istiod-ca-cert\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:06.985967 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.985915 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74tzq\" (UniqueName: \"kubernetes.io/projected/19c025e8-48e4-4a4c-9411-cd009a7e321c-kube-api-access-74tzq\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:06.985967 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.985948 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-data\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:06.986081 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.985988 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-workload-socket\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:06.999784 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:06.999749 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8"] Jul 07 06:57:07.030780 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.030736 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:07.086940 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.086910 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19c025e8-48e4-4a4c-9411-cd009a7e321c-istiod-ca-cert\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.087129 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.086942 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74tzq\" (UniqueName: \"kubernetes.io/projected/19c025e8-48e4-4a4c-9411-cd009a7e321c-kube-api-access-74tzq\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.087129 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.086974 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-data\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.087129 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.087004 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-workload-socket\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.087129 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.087020 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-credential-socket\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.087129 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.087035 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-workload-certs\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.087129 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.087061 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-envoy\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.087129 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.087081 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-podinfo\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.087129 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.087105 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-token\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.087586 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.087456 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-workload-certs\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.087586 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.087485 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-workload-socket\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.087586 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.087543 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-data\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.087753 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.087606 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-credential-socket\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.087753 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.087668 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19c025e8-48e4-4a4c-9411-cd009a7e321c-istiod-ca-cert\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.089477 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.089455 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-envoy\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.089773 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.089755 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-podinfo\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.102074 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.102043 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-token\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.102783 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.102763 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74tzq\" (UniqueName: \"kubernetes.io/projected/19c025e8-48e4-4a4c-9411-cd009a7e321c-kube-api-access-74tzq\") pod \"e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.164976 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.164941 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:07.279575 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.279545 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:07.416955 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.416927 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8"] Jul 07 06:57:07.417062 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:57:07.416979 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19c025e8_48e4_4a4c_9411_cd009a7e321c.slice/crio-7d9f66a65e18d13aa4d580765a91b7fe154f5adccf149ec5e83a195ef37b8f1d WatchSource:0}: Error finding container 7d9f66a65e18d13aa4d580765a91b7fe154f5adccf149ec5e83a195ef37b8f1d: Status 404 returned error can't find the container with id 7d9f66a65e18d13aa4d580765a91b7fe154f5adccf149ec5e83a195ef37b8f1d Jul 07 06:57:07.419129 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.419103 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:57:07.419213 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.419164 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:57:07.419213 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:07.419194 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:57:08.153621 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.153590 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:08.215840 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.215807 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" event={"ID":"19c025e8-48e4-4a4c-9411-cd009a7e321c","Type":"ContainerStarted","Data":"9210de1e7200a56ebc9b76d79e01aecbaf6684f5f5e3c7d6ecdbb2c4638517aa"} Jul 07 06:57:08.216000 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.215863 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" event={"ID":"19c025e8-48e4-4a4c-9411-cd009a7e321c","Type":"ContainerStarted","Data":"7d9f66a65e18d13aa4d580765a91b7fe154f5adccf149ec5e83a195ef37b8f1d"} Jul 07 06:57:08.235939 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.235906 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn"] Jul 07 06:57:08.238850 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.238826 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" Jul 07 06:57:08.243903 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.243857 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-2905275a-serving-cert\"" Jul 07 06:57:08.244020 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.243986 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-754v8\"" Jul 07 06:57:08.249984 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.249941 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" podStartSLOduration=2.249927513 podStartE2EDuration="2.249927513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:57:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:57:08.248145358 +0000 UTC m=+1504.087986326" watchObservedRunningTime="2026-07-07 06:57:08.249927513 +0000 UTC m=+1504.089768479" Jul 07 06:57:08.264293 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.264271 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn"] Jul 07 06:57:08.274764 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.274741 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:08.280023 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.280005 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:08.284858 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.284835 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:08.298035 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.298004 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdb9r\" (UniqueName: \"kubernetes.io/projected/c3aca146-8923-4233-b720-0aba3820c1c3-kube-api-access-gdb9r\") pod \"maas-api-e2e-derive-2905275a-847c554b6d-m52pn\" (UID: \"c3aca146-8923-4233-b720-0aba3820c1c3\") " pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" Jul 07 06:57:08.298141 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.298070 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3aca146-8923-4233-b720-0aba3820c1c3-openshift-service-ca\") pod \"maas-api-e2e-derive-2905275a-847c554b6d-m52pn\" (UID: \"c3aca146-8923-4233-b720-0aba3820c1c3\") " pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" Jul 07 06:57:08.298187 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.298169 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3aca146-8923-4233-b720-0aba3820c1c3-maas-api-tls\") pod \"maas-api-e2e-derive-2905275a-847c554b6d-m52pn\" (UID: \"c3aca146-8923-4233-b720-0aba3820c1c3\") " pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" Jul 07 06:57:08.398593 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.398561 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdb9r\" (UniqueName: \"kubernetes.io/projected/c3aca146-8923-4233-b720-0aba3820c1c3-kube-api-access-gdb9r\") pod \"maas-api-e2e-derive-2905275a-847c554b6d-m52pn\" (UID: \"c3aca146-8923-4233-b720-0aba3820c1c3\") " pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" Jul 07 06:57:08.398754 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.398642 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3aca146-8923-4233-b720-0aba3820c1c3-openshift-service-ca\") pod \"maas-api-e2e-derive-2905275a-847c554b6d-m52pn\" (UID: \"c3aca146-8923-4233-b720-0aba3820c1c3\") " pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" Jul 07 06:57:08.398754 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.398732 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3aca146-8923-4233-b720-0aba3820c1c3-maas-api-tls\") pod \"maas-api-e2e-derive-2905275a-847c554b6d-m52pn\" (UID: \"c3aca146-8923-4233-b720-0aba3820c1c3\") " pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" Jul 07 06:57:08.399300 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.399271 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3aca146-8923-4233-b720-0aba3820c1c3-openshift-service-ca\") pod \"maas-api-e2e-derive-2905275a-847c554b6d-m52pn\" (UID: \"c3aca146-8923-4233-b720-0aba3820c1c3\") " pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" Jul 07 06:57:08.401026 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.401008 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3aca146-8923-4233-b720-0aba3820c1c3-maas-api-tls\") pod \"maas-api-e2e-derive-2905275a-847c554b6d-m52pn\" (UID: \"c3aca146-8923-4233-b720-0aba3820c1c3\") " pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" Jul 07 06:57:08.419399 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.419331 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdb9r\" (UniqueName: \"kubernetes.io/projected/c3aca146-8923-4233-b720-0aba3820c1c3-kube-api-access-gdb9r\") pod \"maas-api-e2e-derive-2905275a-847c554b6d-m52pn\" (UID: \"c3aca146-8923-4233-b720-0aba3820c1c3\") " pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" Jul 07 06:57:08.445146 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.445116 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:08.550349 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.550322 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" Jul 07 06:57:08.690284 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.690190 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:08.697388 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.697341 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn"] Jul 07 06:57:08.698630 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:57:08.698607 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3aca146_8923_4233_b720_0aba3820c1c3.slice/crio-832f06d5e077c57d1b766d794b218ed629ab3f02f35d962d53a7148f878aa661 WatchSource:0}: Error finding container 832f06d5e077c57d1b766d794b218ed629ab3f02f35d962d53a7148f878aa661: Status 404 returned error can't find the container with id 832f06d5e077c57d1b766d794b218ed629ab3f02f35d962d53a7148f878aa661 Jul 07 06:57:08.998764 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:08.998658 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:09.099597 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:09.099564 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:09.222169 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:09.222121 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" event={"ID":"c3aca146-8923-4233-b720-0aba3820c1c3","Type":"ContainerStarted","Data":"1cbe978780554b7e6a5d2cd2276c9315a384870d266d374aaa189d19c6eeec77"} Jul 07 06:57:09.222622 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:09.222186 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" event={"ID":"c3aca146-8923-4233-b720-0aba3820c1c3","Type":"ContainerStarted","Data":"832f06d5e077c57d1b766d794b218ed629ab3f02f35d962d53a7148f878aa661"} Jul 07 06:57:09.222622 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:09.222391 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:09.222622 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:09.222440 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" Jul 07 06:57:09.223419 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:09.223396 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:09.245281 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:09.245234 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" podStartSLOduration=0.930324716 podStartE2EDuration="1.245217382s" totalImagesPullingTime="314.892666ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:57:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:57:09.24324099 +0000 UTC m=+1505.083081958" watchObservedRunningTime="2026-07-07 06:57:09.245217382 +0000 UTC m=+1505.085058349" Jul 07 06:57:10.222572 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:10.222534 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:12.520261 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:12.520211 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:12.663441 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:12.663402 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8"] Jul 07 06:57:12.663633 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:12.663575 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" podUID="19c025e8-48e4-4a4c-9411-cd009a7e321c" containerName="istio-proxy" containerID="cri-o://9210de1e7200a56ebc9b76d79e01aecbaf6684f5f5e3c7d6ecdbb2c4638517aa" gracePeriod=30 Jul 07 06:57:12.694831 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:12.694799 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:12.794060 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:12.794022 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:12.903056 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:12.903024 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn"] Jul 07 06:57:12.903247 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:12.903219 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" podUID="c3aca146-8923-4233-b720-0aba3820c1c3" containerName="maas-api" containerID="cri-o://1cbe978780554b7e6a5d2cd2276c9315a384870d266d374aaa189d19c6eeec77" gracePeriod=30 Jul 07 06:57:12.927746 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:12.927711 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:13.179652 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.179629 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" Jul 07 06:57:13.254016 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.253991 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdb9r\" (UniqueName: \"kubernetes.io/projected/c3aca146-8923-4233-b720-0aba3820c1c3-kube-api-access-gdb9r\") pod \"c3aca146-8923-4233-b720-0aba3820c1c3\" (UID: \"c3aca146-8923-4233-b720-0aba3820c1c3\") " Jul 07 06:57:13.254016 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.254026 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3aca146-8923-4233-b720-0aba3820c1c3-openshift-service-ca\") pod \"c3aca146-8923-4233-b720-0aba3820c1c3\" (UID: \"c3aca146-8923-4233-b720-0aba3820c1c3\") " Jul 07 06:57:13.254275 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.254150 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3aca146-8923-4233-b720-0aba3820c1c3-maas-api-tls\") pod \"c3aca146-8923-4233-b720-0aba3820c1c3\" (UID: \"c3aca146-8923-4233-b720-0aba3820c1c3\") " Jul 07 06:57:13.254707 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.254663 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3aca146-8923-4233-b720-0aba3820c1c3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c3aca146-8923-4233-b720-0aba3820c1c3" (UID: "c3aca146-8923-4233-b720-0aba3820c1c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 06:57:13.256480 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.256451 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3aca146-8923-4233-b720-0aba3820c1c3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c3aca146-8923-4233-b720-0aba3820c1c3" (UID: "c3aca146-8923-4233-b720-0aba3820c1c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 06:57:13.256757 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.256735 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3aca146-8923-4233-b720-0aba3820c1c3-kube-api-access-gdb9r" (OuterVolumeSpecName: "kube-api-access-gdb9r") pod "c3aca146-8923-4233-b720-0aba3820c1c3" (UID: "c3aca146-8923-4233-b720-0aba3820c1c3"). InnerVolumeSpecName "kube-api-access-gdb9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:57:13.261154 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.261130 2579 generic.go:356] "Generic (PLEG): container finished" podID="c3aca146-8923-4233-b720-0aba3820c1c3" containerID="1cbe978780554b7e6a5d2cd2276c9315a384870d266d374aaa189d19c6eeec77" exitCode=0 Jul 07 06:57:13.261239 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.261178 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" Jul 07 06:57:13.261239 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.261206 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" event={"ID":"c3aca146-8923-4233-b720-0aba3820c1c3","Type":"ContainerDied","Data":"1cbe978780554b7e6a5d2cd2276c9315a384870d266d374aaa189d19c6eeec77"} Jul 07 06:57:13.261309 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.261254 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn" event={"ID":"c3aca146-8923-4233-b720-0aba3820c1c3","Type":"ContainerDied","Data":"832f06d5e077c57d1b766d794b218ed629ab3f02f35d962d53a7148f878aa661"} Jul 07 06:57:13.261309 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.261279 2579 scope.go:117] "RemoveContainer" containerID="1cbe978780554b7e6a5d2cd2276c9315a384870d266d374aaa189d19c6eeec77" Jul 07 06:57:13.273558 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.273541 2579 scope.go:117] "RemoveContainer" containerID="1cbe978780554b7e6a5d2cd2276c9315a384870d266d374aaa189d19c6eeec77" Jul 07 06:57:13.273812 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:57:13.273792 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cbe978780554b7e6a5d2cd2276c9315a384870d266d374aaa189d19c6eeec77\": container with ID starting with 1cbe978780554b7e6a5d2cd2276c9315a384870d266d374aaa189d19c6eeec77 not found: ID does not exist" containerID="1cbe978780554b7e6a5d2cd2276c9315a384870d266d374aaa189d19c6eeec77" Jul 07 06:57:13.273870 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.273819 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cbe978780554b7e6a5d2cd2276c9315a384870d266d374aaa189d19c6eeec77"} err="failed to get container status \"1cbe978780554b7e6a5d2cd2276c9315a384870d266d374aaa189d19c6eeec77\": rpc error: code = NotFound desc = could not find container \"1cbe978780554b7e6a5d2cd2276c9315a384870d266d374aaa189d19c6eeec77\": container with ID starting with 1cbe978780554b7e6a5d2cd2276c9315a384870d266d374aaa189d19c6eeec77 not found: ID does not exist" Jul 07 06:57:13.286266 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.286241 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn"] Jul 07 06:57:13.291036 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.291016 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-2905275a-847c554b6d-m52pn"] Jul 07 06:57:13.355477 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.355398 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3aca146-8923-4233-b720-0aba3820c1c3-maas-api-tls\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:13.355477 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.355429 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdb9r\" (UniqueName: \"kubernetes.io/projected/c3aca146-8923-4233-b720-0aba3820c1c3-kube-api-access-gdb9r\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:13.355477 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:13.355442 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3aca146-8923-4233-b720-0aba3820c1c3-openshift-service-ca\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:14.293598 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:14.293554 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:14.783615 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:14.783576 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3aca146-8923-4233-b720-0aba3820c1c3" path="/var/lib/kubelet/pods/c3aca146-8923-4233-b720-0aba3820c1c3/volumes" Jul 07 06:57:18.724914 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.724858 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:18.808682 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.808653 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-credential-socket\") pod \"19c025e8-48e4-4a4c-9411-cd009a7e321c\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " Jul 07 06:57:18.808836 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.808692 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19c025e8-48e4-4a4c-9411-cd009a7e321c-istiod-ca-cert\") pod \"19c025e8-48e4-4a4c-9411-cd009a7e321c\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " Jul 07 06:57:18.808836 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.808709 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-data\") pod \"19c025e8-48e4-4a4c-9411-cd009a7e321c\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " Jul 07 06:57:18.808836 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.808731 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-token\") pod \"19c025e8-48e4-4a4c-9411-cd009a7e321c\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " Jul 07 06:57:18.808836 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.808747 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-podinfo\") pod \"19c025e8-48e4-4a4c-9411-cd009a7e321c\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " Jul 07 06:57:18.808836 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.808793 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-workload-socket\") pod \"19c025e8-48e4-4a4c-9411-cd009a7e321c\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " Jul 07 06:57:18.809112 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.808851 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74tzq\" (UniqueName: \"kubernetes.io/projected/19c025e8-48e4-4a4c-9411-cd009a7e321c-kube-api-access-74tzq\") pod \"19c025e8-48e4-4a4c-9411-cd009a7e321c\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " Jul 07 06:57:18.809112 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.808909 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-workload-certs\") pod \"19c025e8-48e4-4a4c-9411-cd009a7e321c\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " Jul 07 06:57:18.809112 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.808932 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-envoy\") pod \"19c025e8-48e4-4a4c-9411-cd009a7e321c\" (UID: \"19c025e8-48e4-4a4c-9411-cd009a7e321c\") " Jul 07 06:57:18.809112 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.808976 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "19c025e8-48e4-4a4c-9411-cd009a7e321c" (UID: "19c025e8-48e4-4a4c-9411-cd009a7e321c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:57:18.809112 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.809104 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19c025e8-48e4-4a4c-9411-cd009a7e321c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "19c025e8-48e4-4a4c-9411-cd009a7e321c" (UID: "19c025e8-48e4-4a4c-9411-cd009a7e321c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 06:57:18.809420 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.809206 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "19c025e8-48e4-4a4c-9411-cd009a7e321c" (UID: "19c025e8-48e4-4a4c-9411-cd009a7e321c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:57:18.809420 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.809244 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-credential-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:18.809420 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.809264 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19c025e8-48e4-4a4c-9411-cd009a7e321c-istiod-ca-cert\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:18.809420 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.809325 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "19c025e8-48e4-4a4c-9411-cd009a7e321c" (UID: "19c025e8-48e4-4a4c-9411-cd009a7e321c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:57:18.809645 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.809612 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-data" (OuterVolumeSpecName: "istio-data") pod "19c025e8-48e4-4a4c-9411-cd009a7e321c" (UID: "19c025e8-48e4-4a4c-9411-cd009a7e321c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:57:18.811041 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.811006 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19c025e8-48e4-4a4c-9411-cd009a7e321c-kube-api-access-74tzq" (OuterVolumeSpecName: "kube-api-access-74tzq") pod "19c025e8-48e4-4a4c-9411-cd009a7e321c" (UID: "19c025e8-48e4-4a4c-9411-cd009a7e321c"). InnerVolumeSpecName "kube-api-access-74tzq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:57:18.811160 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.811085 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "19c025e8-48e4-4a4c-9411-cd009a7e321c" (UID: "19c025e8-48e4-4a4c-9411-cd009a7e321c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 06:57:18.811284 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.811268 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-token" (OuterVolumeSpecName: "istio-token") pod "19c025e8-48e4-4a4c-9411-cd009a7e321c" (UID: "19c025e8-48e4-4a4c-9411-cd009a7e321c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:57:18.811330 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.811311 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "19c025e8-48e4-4a4c-9411-cd009a7e321c" (UID: "19c025e8-48e4-4a4c-9411-cd009a7e321c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:57:18.910448 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.910422 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-data\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:18.910448 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.910442 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-token\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:18.910448 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.910451 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-podinfo\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:18.910655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.910459 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-workload-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:18.910655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.910468 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74tzq\" (UniqueName: \"kubernetes.io/projected/19c025e8-48e4-4a4c-9411-cd009a7e321c-kube-api-access-74tzq\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:18.910655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.910477 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-workload-certs\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:18.910655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:18.910485 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19c025e8-48e4-4a4c-9411-cd009a7e321c-istio-envoy\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:57:19.269186 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:19.269153 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:19.300359 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:19.300327 2579 generic.go:356] "Generic (PLEG): container finished" podID="19c025e8-48e4-4a4c-9411-cd009a7e321c" containerID="9210de1e7200a56ebc9b76d79e01aecbaf6684f5f5e3c7d6ecdbb2c4638517aa" exitCode=0 Jul 07 06:57:19.300535 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:19.300389 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" Jul 07 06:57:19.300535 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:19.300410 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" event={"ID":"19c025e8-48e4-4a4c-9411-cd009a7e321c","Type":"ContainerDied","Data":"9210de1e7200a56ebc9b76d79e01aecbaf6684f5f5e3c7d6ecdbb2c4638517aa"} Jul 07 06:57:19.300535 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:19.300464 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8" event={"ID":"19c025e8-48e4-4a4c-9411-cd009a7e321c","Type":"ContainerDied","Data":"7d9f66a65e18d13aa4d580765a91b7fe154f5adccf149ec5e83a195ef37b8f1d"} Jul 07 06:57:19.300535 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:19.300494 2579 scope.go:117] "RemoveContainer" containerID="9210de1e7200a56ebc9b76d79e01aecbaf6684f5f5e3c7d6ecdbb2c4638517aa" Jul 07 06:57:19.310040 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:19.310023 2579 scope.go:117] "RemoveContainer" containerID="9210de1e7200a56ebc9b76d79e01aecbaf6684f5f5e3c7d6ecdbb2c4638517aa" Jul 07 06:57:19.310280 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:57:19.310263 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9210de1e7200a56ebc9b76d79e01aecbaf6684f5f5e3c7d6ecdbb2c4638517aa\": container with ID starting with 9210de1e7200a56ebc9b76d79e01aecbaf6684f5f5e3c7d6ecdbb2c4638517aa not found: ID does not exist" containerID="9210de1e7200a56ebc9b76d79e01aecbaf6684f5f5e3c7d6ecdbb2c4638517aa" Jul 07 06:57:19.310326 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:19.310288 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9210de1e7200a56ebc9b76d79e01aecbaf6684f5f5e3c7d6ecdbb2c4638517aa"} err="failed to get container status \"9210de1e7200a56ebc9b76d79e01aecbaf6684f5f5e3c7d6ecdbb2c4638517aa\": rpc error: code = NotFound desc = could not find container \"9210de1e7200a56ebc9b76d79e01aecbaf6684f5f5e3c7d6ecdbb2c4638517aa\": container with ID starting with 9210de1e7200a56ebc9b76d79e01aecbaf6684f5f5e3c7d6ecdbb2c4638517aa not found: ID does not exist" Jul 07 06:57:19.327698 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:19.327666 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8"] Jul 07 06:57:19.338008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:19.333782 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-2905275a-openshift-default-6c675877dc-4nmt8"] Jul 07 06:57:19.368919 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:19.368889 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:19.759335 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:19.759302 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:19.859047 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:19.859015 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:19.960768 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:19.960737 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:20.783128 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:20.783096 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19c025e8-48e4-4a4c-9411-cd009a7e321c" path="/var/lib/kubelet/pods/19c025e8-48e4-4a4c-9411-cd009a7e321c/volumes" Jul 07 06:57:21.350336 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:21.350298 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:27.085579 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:27.085485 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:27.270015 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:27.269974 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:27.379403 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:27.379291 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:27.490402 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:27.490344 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:27.588989 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:27.588944 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:28.005936 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:28.005904 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:57:54.016104 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:57:54.016064 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:00.232034 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:00.231988 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:00.336136 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:00.336099 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:00.434060 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:00.434026 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:16.395601 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:16.395564 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:22.528837 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:22.528768 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:22.637184 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:22.637146 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:22.729857 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:22.729820 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:33.313408 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:33.313355 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:34.186607 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:34.186569 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:40.604873 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:40.604828 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:40.796822 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:40.796789 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:40.943385 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:40.943279 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:41.774069 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:41.774038 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:41.862550 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:41.862519 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:41.962683 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:41.962650 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:43.633817 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:43.633780 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:49.657822 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:49.657784 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:49.804398 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:49.804346 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:50.383769 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:50.383728 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:50.621633 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:50.621594 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:50.689765 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:50.689681 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:56.627160 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:56.627071 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:56.728202 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:56.728165 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:56.901570 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:56.901469 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:57.211719 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.211638 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w"] Jul 07 06:58:57.212001 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.211989 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19c025e8-48e4-4a4c-9411-cd009a7e321c" containerName="istio-proxy" Jul 07 06:58:57.212044 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.212003 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="19c025e8-48e4-4a4c-9411-cd009a7e321c" containerName="istio-proxy" Jul 07 06:58:57.212044 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.212026 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3aca146-8923-4233-b720-0aba3820c1c3" containerName="maas-api" Jul 07 06:58:57.212044 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.212031 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3aca146-8923-4233-b720-0aba3820c1c3" containerName="maas-api" Jul 07 06:58:57.212140 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.212093 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3aca146-8923-4233-b720-0aba3820c1c3" containerName="maas-api" Jul 07 06:58:57.212140 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.212105 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="19c025e8-48e4-4a4c-9411-cd009a7e321c" containerName="istio-proxy" Jul 07 06:58:57.221202 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.221178 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.224296 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.224273 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-811cf297-openshift-default-dockercfg-zszvq\"" Jul 07 06:58:57.237072 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.237048 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w"] Jul 07 06:58:57.245115 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.245086 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:57.321954 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.321920 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:58:57.382782 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.382748 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-token\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.382952 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.382790 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49k44\" (UniqueName: \"kubernetes.io/projected/f87dba03-c6c9-4632-bb1e-e60c803c885c-kube-api-access-49k44\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.382952 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.382882 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-workload-certs\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.382952 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.382931 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-workload-socket\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.383120 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.383006 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-envoy\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.383120 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.383034 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f87dba03-c6c9-4632-bb1e-e60c803c885c-istiod-ca-cert\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.383120 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.383061 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-data\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.383120 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.383112 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-credential-socket\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.383301 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.383201 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-podinfo\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.484028 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.483939 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-token\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.484028 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.483974 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49k44\" (UniqueName: \"kubernetes.io/projected/f87dba03-c6c9-4632-bb1e-e60c803c885c-kube-api-access-49k44\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.484028 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.484010 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-workload-certs\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.484310 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.484052 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-workload-socket\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.484310 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.484091 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-envoy\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.484310 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.484201 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f87dba03-c6c9-4632-bb1e-e60c803c885c-istiod-ca-cert\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.484310 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.484230 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-data\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.484310 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.484276 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-credential-socket\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.484620 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.484405 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-podinfo\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.484620 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.484452 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-workload-certs\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.484620 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.484552 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-workload-socket\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.484785 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.484765 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-data\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.484785 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.484774 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-credential-socket\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.485014 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.484990 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f87dba03-c6c9-4632-bb1e-e60c803c885c-istiod-ca-cert\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.486468 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.486446 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-envoy\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.486793 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.486778 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-podinfo\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.495038 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.495017 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-token\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.495163 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.495097 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49k44\" (UniqueName: \"kubernetes.io/projected/f87dba03-c6c9-4632-bb1e-e60c803c885c-kube-api-access-49k44\") pod \"e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.532219 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.532195 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:57.668309 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.668175 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w"] Jul 07 06:58:57.671253 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:58:57.671222 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf87dba03_c6c9_4632_bb1e_e60c803c885c.slice/crio-3bdf1993185807c6cc437c0642cb73b964c8456fdd26bb21846f81608ed20f6d WatchSource:0}: Error finding container 3bdf1993185807c6cc437c0642cb73b964c8456fdd26bb21846f81608ed20f6d: Status 404 returned error can't find the container with id 3bdf1993185807c6cc437c0642cb73b964c8456fdd26bb21846f81608ed20f6d Jul 07 06:58:57.673430 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.673396 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:58:57.673507 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.673468 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:58:57.673507 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.673499 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:58:57.810256 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.810217 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" event={"ID":"f87dba03-c6c9-4632-bb1e-e60c803c885c","Type":"ContainerStarted","Data":"250e517ba9982d2ac951385f23473cd92fe4dde4022a5f664129d8a08a25dd9c"} Jul 07 06:58:57.810452 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.810272 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" event={"ID":"f87dba03-c6c9-4632-bb1e-e60c803c885c","Type":"ContainerStarted","Data":"3bdf1993185807c6cc437c0642cb73b964c8456fdd26bb21846f81608ed20f6d"} Jul 07 06:58:57.837401 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:57.837339 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" podStartSLOduration=0.83732156 podStartE2EDuration="837.32156ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:58:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:58:57.836279443 +0000 UTC m=+1613.676120415" watchObservedRunningTime="2026-07-07 06:58:57.83732156 +0000 UTC m=+1613.677162530" Jul 07 06:58:58.532516 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:58.532486 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:58.537179 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:58.537156 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:58.815595 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:58.815509 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:58:58.816460 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:58:58.816440 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:59:00.213599 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.213561 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:00.273664 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.273626 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl"] Jul 07 06:59:00.277444 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.277420 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:00.281277 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.281250 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-754v8\"" Jul 07 06:59:00.281277 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.281266 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-811cf297-serving-cert\"" Jul 07 06:59:00.291568 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.291545 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl"] Jul 07 06:59:00.314414 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.314382 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:00.411905 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.411868 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de0e01ac-c521-46ba-90ec-42caf0e6faed-maas-api-tls\") pod \"maas-api-e2e-mt-811cf297-578bff9958-gccxl\" (UID: \"de0e01ac-c521-46ba-90ec-42caf0e6faed\") " pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:00.411905 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.411904 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfr5c\" (UniqueName: \"kubernetes.io/projected/de0e01ac-c521-46ba-90ec-42caf0e6faed-kube-api-access-bfr5c\") pod \"maas-api-e2e-mt-811cf297-578bff9958-gccxl\" (UID: \"de0e01ac-c521-46ba-90ec-42caf0e6faed\") " pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:00.412102 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.411941 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de0e01ac-c521-46ba-90ec-42caf0e6faed-openshift-service-ca\") pod \"maas-api-e2e-mt-811cf297-578bff9958-gccxl\" (UID: \"de0e01ac-c521-46ba-90ec-42caf0e6faed\") " pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:00.475780 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.475684 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:00.512542 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.512511 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de0e01ac-c521-46ba-90ec-42caf0e6faed-maas-api-tls\") pod \"maas-api-e2e-mt-811cf297-578bff9958-gccxl\" (UID: \"de0e01ac-c521-46ba-90ec-42caf0e6faed\") " pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:00.512719 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.512546 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfr5c\" (UniqueName: \"kubernetes.io/projected/de0e01ac-c521-46ba-90ec-42caf0e6faed-kube-api-access-bfr5c\") pod \"maas-api-e2e-mt-811cf297-578bff9958-gccxl\" (UID: \"de0e01ac-c521-46ba-90ec-42caf0e6faed\") " pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:00.512719 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.512605 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de0e01ac-c521-46ba-90ec-42caf0e6faed-openshift-service-ca\") pod \"maas-api-e2e-mt-811cf297-578bff9958-gccxl\" (UID: \"de0e01ac-c521-46ba-90ec-42caf0e6faed\") " pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:00.512719 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:59:00.512671 2579 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-811cf297-serving-cert: secret "maas-api-e2e-mt-811cf297-serving-cert" not found Jul 07 06:59:00.512892 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:59:00.512727 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0e01ac-c521-46ba-90ec-42caf0e6faed-maas-api-tls podName:de0e01ac-c521-46ba-90ec-42caf0e6faed nodeName:}" failed. No retries permitted until 2026-07-07 06:59:01.012711297 +0000 UTC m=+1616.852552241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/de0e01ac-c521-46ba-90ec-42caf0e6faed-maas-api-tls") pod "maas-api-e2e-mt-811cf297-578bff9958-gccxl" (UID: "de0e01ac-c521-46ba-90ec-42caf0e6faed") : secret "maas-api-e2e-mt-811cf297-serving-cert" not found Jul 07 06:59:00.513277 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.513254 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de0e01ac-c521-46ba-90ec-42caf0e6faed-openshift-service-ca\") pod \"maas-api-e2e-mt-811cf297-578bff9958-gccxl\" (UID: \"de0e01ac-c521-46ba-90ec-42caf0e6faed\") " pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:00.527430 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.527405 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfr5c\" (UniqueName: \"kubernetes.io/projected/de0e01ac-c521-46ba-90ec-42caf0e6faed-kube-api-access-bfr5c\") pod \"maas-api-e2e-mt-811cf297-578bff9958-gccxl\" (UID: \"de0e01ac-c521-46ba-90ec-42caf0e6faed\") " pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:00.771586 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.771504 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:00.969081 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:00.969050 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:01.017278 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:01.017248 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de0e01ac-c521-46ba-90ec-42caf0e6faed-maas-api-tls\") pod \"maas-api-e2e-mt-811cf297-578bff9958-gccxl\" (UID: \"de0e01ac-c521-46ba-90ec-42caf0e6faed\") " pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:01.019865 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:01.019831 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de0e01ac-c521-46ba-90ec-42caf0e6faed-maas-api-tls\") pod \"maas-api-e2e-mt-811cf297-578bff9958-gccxl\" (UID: \"de0e01ac-c521-46ba-90ec-42caf0e6faed\") " pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:01.070833 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:01.070805 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:01.191136 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:01.191097 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:01.335965 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:01.335883 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl"] Jul 07 06:59:01.338839 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:59:01.338808 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde0e01ac_c521_46ba_90ec_42caf0e6faed.slice/crio-4a97023ce0f5fd6d9ee31daf7afa73e00033e8e741999cd59daaeefcd9c9a4ea WatchSource:0}: Error finding container 4a97023ce0f5fd6d9ee31daf7afa73e00033e8e741999cd59daaeefcd9c9a4ea: Status 404 returned error can't find the container with id 4a97023ce0f5fd6d9ee31daf7afa73e00033e8e741999cd59daaeefcd9c9a4ea Jul 07 06:59:01.831009 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:01.830972 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" event={"ID":"de0e01ac-c521-46ba-90ec-42caf0e6faed","Type":"ContainerStarted","Data":"4a97023ce0f5fd6d9ee31daf7afa73e00033e8e741999cd59daaeefcd9c9a4ea"} Jul 07 06:59:02.837406 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:02.837352 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" event={"ID":"de0e01ac-c521-46ba-90ec-42caf0e6faed","Type":"ContainerStarted","Data":"130a28b2e9a8059e91024acdf976b0bb855c95dbaee0290901fe88f75fd45fb7"} Jul 07 06:59:02.837784 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:02.837490 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:02.861567 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:02.861510 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" podStartSLOduration=2.32135096 podStartE2EDuration="2.861496408s" totalImagesPullingTime="540.145448ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:59:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:02.859962403 +0000 UTC m=+1618.699803371" watchObservedRunningTime="2026-07-07 06:59:02.861496408 +0000 UTC m=+1618.701337375" Jul 07 06:59:04.699357 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:04.699316 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:05.366747 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:05.366711 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:05.464706 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:05.464663 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:05.817145 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:05.817104 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:08.857294 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:08.857260 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:12.134529 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:12.134462 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:12.252240 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:12.252198 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:12.579765 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:12.579730 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:12.867600 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:12.867516 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl"] Jul 07 06:59:12.867789 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:12.867756 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" podUID="de0e01ac-c521-46ba-90ec-42caf0e6faed" containerName="maas-api" containerID="cri-o://130a28b2e9a8059e91024acdf976b0bb855c95dbaee0290901fe88f75fd45fb7" gracePeriod=30 Jul 07 06:59:12.897941 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:12.897884 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:12.996572 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:12.996530 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:13.126877 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.126817 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:13.224379 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.224329 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de0e01ac-c521-46ba-90ec-42caf0e6faed-openshift-service-ca\") pod \"de0e01ac-c521-46ba-90ec-42caf0e6faed\" (UID: \"de0e01ac-c521-46ba-90ec-42caf0e6faed\") " Jul 07 06:59:13.224856 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.224410 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de0e01ac-c521-46ba-90ec-42caf0e6faed-maas-api-tls\") pod \"de0e01ac-c521-46ba-90ec-42caf0e6faed\" (UID: \"de0e01ac-c521-46ba-90ec-42caf0e6faed\") " Jul 07 06:59:13.224856 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.224485 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfr5c\" (UniqueName: \"kubernetes.io/projected/de0e01ac-c521-46ba-90ec-42caf0e6faed-kube-api-access-bfr5c\") pod \"de0e01ac-c521-46ba-90ec-42caf0e6faed\" (UID: \"de0e01ac-c521-46ba-90ec-42caf0e6faed\") " Jul 07 06:59:13.224856 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.224755 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de0e01ac-c521-46ba-90ec-42caf0e6faed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "de0e01ac-c521-46ba-90ec-42caf0e6faed" (UID: "de0e01ac-c521-46ba-90ec-42caf0e6faed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 06:59:13.226595 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.226570 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de0e01ac-c521-46ba-90ec-42caf0e6faed-kube-api-access-bfr5c" (OuterVolumeSpecName: "kube-api-access-bfr5c") pod "de0e01ac-c521-46ba-90ec-42caf0e6faed" (UID: "de0e01ac-c521-46ba-90ec-42caf0e6faed"). InnerVolumeSpecName "kube-api-access-bfr5c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:59:13.226882 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.226852 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de0e01ac-c521-46ba-90ec-42caf0e6faed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "de0e01ac-c521-46ba-90ec-42caf0e6faed" (UID: "de0e01ac-c521-46ba-90ec-42caf0e6faed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 06:59:13.325377 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.325345 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de0e01ac-c521-46ba-90ec-42caf0e6faed-maas-api-tls\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:59:13.325377 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.325379 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfr5c\" (UniqueName: \"kubernetes.io/projected/de0e01ac-c521-46ba-90ec-42caf0e6faed-kube-api-access-bfr5c\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:59:13.325572 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.325389 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de0e01ac-c521-46ba-90ec-42caf0e6faed-openshift-service-ca\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:59:13.907664 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.907631 2579 generic.go:356] "Generic (PLEG): container finished" podID="de0e01ac-c521-46ba-90ec-42caf0e6faed" containerID="130a28b2e9a8059e91024acdf976b0bb855c95dbaee0290901fe88f75fd45fb7" exitCode=0 Jul 07 06:59:13.907934 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.907677 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" Jul 07 06:59:13.907934 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.907721 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" event={"ID":"de0e01ac-c521-46ba-90ec-42caf0e6faed","Type":"ContainerDied","Data":"130a28b2e9a8059e91024acdf976b0bb855c95dbaee0290901fe88f75fd45fb7"} Jul 07 06:59:13.907934 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.907778 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl" event={"ID":"de0e01ac-c521-46ba-90ec-42caf0e6faed","Type":"ContainerDied","Data":"4a97023ce0f5fd6d9ee31daf7afa73e00033e8e741999cd59daaeefcd9c9a4ea"} Jul 07 06:59:13.907934 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.907807 2579 scope.go:117] "RemoveContainer" containerID="130a28b2e9a8059e91024acdf976b0bb855c95dbaee0290901fe88f75fd45fb7" Jul 07 06:59:13.917654 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.917631 2579 scope.go:117] "RemoveContainer" containerID="130a28b2e9a8059e91024acdf976b0bb855c95dbaee0290901fe88f75fd45fb7" Jul 07 06:59:13.917915 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:59:13.917898 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"130a28b2e9a8059e91024acdf976b0bb855c95dbaee0290901fe88f75fd45fb7\": container with ID starting with 130a28b2e9a8059e91024acdf976b0bb855c95dbaee0290901fe88f75fd45fb7 not found: ID does not exist" containerID="130a28b2e9a8059e91024acdf976b0bb855c95dbaee0290901fe88f75fd45fb7" Jul 07 06:59:13.918008 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.917921 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"130a28b2e9a8059e91024acdf976b0bb855c95dbaee0290901fe88f75fd45fb7"} err="failed to get container status \"130a28b2e9a8059e91024acdf976b0bb855c95dbaee0290901fe88f75fd45fb7\": rpc error: code = NotFound desc = could not find container \"130a28b2e9a8059e91024acdf976b0bb855c95dbaee0290901fe88f75fd45fb7\": container with ID starting with 130a28b2e9a8059e91024acdf976b0bb855c95dbaee0290901fe88f75fd45fb7 not found: ID does not exist" Jul 07 06:59:13.938220 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.938193 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl"] Jul 07 06:59:13.948684 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:13.948660 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-811cf297-578bff9958-gccxl"] Jul 07 06:59:14.065994 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:14.065111 2579 patch_prober.go:28] interesting pod/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:38118->10.134.0.40:15021: read: connection reset by peer" start-of-body= Jul 07 06:59:14.065994 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:14.065169 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" podUID="f87dba03-c6c9-4632-bb1e-e60c803c885c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:38118->10.134.0.40:15021: read: connection reset by peer" Jul 07 06:59:14.783713 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:14.783669 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de0e01ac-c521-46ba-90ec-42caf0e6faed" path="/var/lib/kubelet/pods/de0e01ac-c521-46ba-90ec-42caf0e6faed/volumes" Jul 07 06:59:14.914759 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:14.914724 2579 generic.go:356] "Generic (PLEG): container finished" podID="f87dba03-c6c9-4632-bb1e-e60c803c885c" containerID="250e517ba9982d2ac951385f23473cd92fe4dde4022a5f664129d8a08a25dd9c" exitCode=0 Jul 07 06:59:14.914934 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:14.914799 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" event={"ID":"f87dba03-c6c9-4632-bb1e-e60c803c885c","Type":"ContainerDied","Data":"250e517ba9982d2ac951385f23473cd92fe4dde4022a5f664129d8a08a25dd9c"} Jul 07 06:59:14.915261 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:14.915233 2579 scope.go:117] "RemoveContainer" containerID="250e517ba9982d2ac951385f23473cd92fe4dde4022a5f664129d8a08a25dd9c" Jul 07 06:59:14.916248 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:14.916219 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:59:14.916353 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:14.916288 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:59:14.916353 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:14.916322 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:59:15.922635 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:15.922602 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" event={"ID":"f87dba03-c6c9-4632-bb1e-e60c803c885c","Type":"ContainerStarted","Data":"45fc0b5206989fc62832b339e1b9e4a2a303651641b9067a48ba3abad305736b"} Jul 07 06:59:16.533070 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:16.533031 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:59:16.533278 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:16.533252 2579 patch_prober.go:28] interesting pod/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 07 06:59:16.533375 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:16.533310 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" podUID="f87dba03-c6c9-4632-bb1e-e60c803c885c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 07 06:59:17.533316 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:17.533281 2579 patch_prober.go:28] interesting pod/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 07 06:59:17.533956 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:17.533923 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" podUID="f87dba03-c6c9-4632-bb1e-e60c803c885c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 07 06:59:18.542847 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:18.542806 2579 patch_prober.go:28] interesting pod/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 07 06:59:18.543461 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:18.543425 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" podUID="f87dba03-c6c9-4632-bb1e-e60c803c885c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 07 06:59:19.085226 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:19.085177 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:19.186986 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:19.186948 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:19.337819 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:19.337706 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w"] Jul 07 06:59:19.338308 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:19.338272 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" podUID="f87dba03-c6c9-4632-bb1e-e60c803c885c" containerName="istio-proxy" containerID="cri-o://45fc0b5206989fc62832b339e1b9e4a2a303651641b9067a48ba3abad305736b" gracePeriod=30 Jul 07 06:59:19.356539 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:19.356433 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:19.592799 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:19.592704 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:19.734534 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:19.734497 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:19.827251 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:19.827215 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:19.940050 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:19.939961 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:24.976895 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:24.976867 2579 generic.go:356] "Generic (PLEG): container finished" podID="f87dba03-c6c9-4632-bb1e-e60c803c885c" containerID="45fc0b5206989fc62832b339e1b9e4a2a303651641b9067a48ba3abad305736b" exitCode=0 Jul 07 06:59:24.977278 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:24.976943 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" event={"ID":"f87dba03-c6c9-4632-bb1e-e60c803c885c","Type":"ContainerDied","Data":"45fc0b5206989fc62832b339e1b9e4a2a303651641b9067a48ba3abad305736b"} Jul 07 06:59:24.977278 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:24.977027 2579 scope.go:117] "RemoveContainer" containerID="250e517ba9982d2ac951385f23473cd92fe4dde4022a5f664129d8a08a25dd9c" Jul 07 06:59:24.996348 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:24.996329 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:59:25.040869 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.040842 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-workload-socket\") pod \"f87dba03-c6c9-4632-bb1e-e60c803c885c\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " Jul 07 06:59:25.041018 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.040883 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f87dba03-c6c9-4632-bb1e-e60c803c885c-istiod-ca-cert\") pod \"f87dba03-c6c9-4632-bb1e-e60c803c885c\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " Jul 07 06:59:25.041018 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.040934 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-envoy\") pod \"f87dba03-c6c9-4632-bb1e-e60c803c885c\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " Jul 07 06:59:25.041018 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.040954 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-data\") pod \"f87dba03-c6c9-4632-bb1e-e60c803c885c\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " Jul 07 06:59:25.041018 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.040990 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-credential-socket\") pod \"f87dba03-c6c9-4632-bb1e-e60c803c885c\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " Jul 07 06:59:25.041255 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.041024 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49k44\" (UniqueName: \"kubernetes.io/projected/f87dba03-c6c9-4632-bb1e-e60c803c885c-kube-api-access-49k44\") pod \"f87dba03-c6c9-4632-bb1e-e60c803c885c\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " Jul 07 06:59:25.041255 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.041050 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-workload-certs\") pod \"f87dba03-c6c9-4632-bb1e-e60c803c885c\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " Jul 07 06:59:25.041255 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.041103 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-token\") pod \"f87dba03-c6c9-4632-bb1e-e60c803c885c\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " Jul 07 06:59:25.041255 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.041119 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f87dba03-c6c9-4632-bb1e-e60c803c885c" (UID: "f87dba03-c6c9-4632-bb1e-e60c803c885c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:59:25.041255 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.041129 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-podinfo\") pod \"f87dba03-c6c9-4632-bb1e-e60c803c885c\" (UID: \"f87dba03-c6c9-4632-bb1e-e60c803c885c\") " Jul 07 06:59:25.041575 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.041359 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f87dba03-c6c9-4632-bb1e-e60c803c885c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f87dba03-c6c9-4632-bb1e-e60c803c885c" (UID: "f87dba03-c6c9-4632-bb1e-e60c803c885c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 06:59:25.041668 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.041645 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f87dba03-c6c9-4632-bb1e-e60c803c885c-istiod-ca-cert\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:59:25.041845 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.041830 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-workload-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:59:25.041994 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.041742 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f87dba03-c6c9-4632-bb1e-e60c803c885c" (UID: "f87dba03-c6c9-4632-bb1e-e60c803c885c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:59:25.042106 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.041809 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f87dba03-c6c9-4632-bb1e-e60c803c885c" (UID: "f87dba03-c6c9-4632-bb1e-e60c803c885c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:59:25.042106 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.041848 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-data" (OuterVolumeSpecName: "istio-data") pod "f87dba03-c6c9-4632-bb1e-e60c803c885c" (UID: "f87dba03-c6c9-4632-bb1e-e60c803c885c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:59:25.043165 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.043142 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f87dba03-c6c9-4632-bb1e-e60c803c885c" (UID: "f87dba03-c6c9-4632-bb1e-e60c803c885c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 06:59:25.043251 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.043147 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f87dba03-c6c9-4632-bb1e-e60c803c885c" (UID: "f87dba03-c6c9-4632-bb1e-e60c803c885c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 06:59:25.043387 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.043349 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f87dba03-c6c9-4632-bb1e-e60c803c885c-kube-api-access-49k44" (OuterVolumeSpecName: "kube-api-access-49k44") pod "f87dba03-c6c9-4632-bb1e-e60c803c885c" (UID: "f87dba03-c6c9-4632-bb1e-e60c803c885c"). InnerVolumeSpecName "kube-api-access-49k44". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:59:25.043833 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.043812 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-token" (OuterVolumeSpecName: "istio-token") pod "f87dba03-c6c9-4632-bb1e-e60c803c885c" (UID: "f87dba03-c6c9-4632-bb1e-e60c803c885c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 06:59:25.142472 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.142358 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-envoy\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:59:25.142472 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.142408 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-data\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:59:25.142472 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.142417 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-credential-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:59:25.142472 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.142426 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49k44\" (UniqueName: \"kubernetes.io/projected/f87dba03-c6c9-4632-bb1e-e60c803c885c-kube-api-access-49k44\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:59:25.142472 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.142434 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f87dba03-c6c9-4632-bb1e-e60c803c885c-workload-certs\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:59:25.142472 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.142442 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-token\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:59:25.142472 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.142449 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f87dba03-c6c9-4632-bb1e-e60c803c885c-istio-podinfo\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 06:59:25.967110 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.967077 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:25.983338 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.983313 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" Jul 07 06:59:25.983717 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.983309 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w" event={"ID":"f87dba03-c6c9-4632-bb1e-e60c803c885c","Type":"ContainerDied","Data":"3bdf1993185807c6cc437c0642cb73b964c8456fdd26bb21846f81608ed20f6d"} Jul 07 06:59:25.983717 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:25.983452 2579 scope.go:117] "RemoveContainer" containerID="45fc0b5206989fc62832b339e1b9e4a2a303651641b9067a48ba3abad305736b" Jul 07 06:59:26.029108 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:26.029078 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w"] Jul 07 06:59:26.034240 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:26.034217 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-811cf297-openshift-default-7f4fd7d7cf-9pl9w"] Jul 07 06:59:26.055134 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:26.055109 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:26.156898 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:26.156846 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:26.331633 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:26.331601 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:26.783135 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:26.783104 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f87dba03-c6c9-4632-bb1e-e60c803c885c" path="/var/lib/kubelet/pods/f87dba03-c6c9-4632-bb1e-e60c803c885c/volumes" Jul 07 06:59:32.675149 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:32.675111 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:32.775867 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:32.775832 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:39.613259 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:39.613201 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:39.939574 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:39.939497 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp"] Jul 07 06:59:39.939868 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:39.939855 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de0e01ac-c521-46ba-90ec-42caf0e6faed" containerName="maas-api" Jul 07 06:59:39.939919 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:39.939868 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="de0e01ac-c521-46ba-90ec-42caf0e6faed" containerName="maas-api" Jul 07 06:59:39.939919 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:39.939879 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f87dba03-c6c9-4632-bb1e-e60c803c885c" containerName="istio-proxy" Jul 07 06:59:39.939919 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:39.939884 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f87dba03-c6c9-4632-bb1e-e60c803c885c" containerName="istio-proxy" Jul 07 06:59:39.939919 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:39.939897 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f87dba03-c6c9-4632-bb1e-e60c803c885c" containerName="istio-proxy" Jul 07 06:59:39.939919 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:39.939902 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f87dba03-c6c9-4632-bb1e-e60c803c885c" containerName="istio-proxy" Jul 07 06:59:39.940110 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:39.939971 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f87dba03-c6c9-4632-bb1e-e60c803c885c" containerName="istio-proxy" Jul 07 06:59:39.940110 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:39.939983 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f87dba03-c6c9-4632-bb1e-e60c803c885c" containerName="istio-proxy" Jul 07 06:59:39.940110 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:39.939992 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="de0e01ac-c521-46ba-90ec-42caf0e6faed" containerName="maas-api" Jul 07 06:59:39.942932 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:39.942917 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:39.946533 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:39.946509 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f08145e7-openshift-default-dockercfg-wd7jp\"" Jul 07 06:59:39.960109 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:39.960085 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp"] Jul 07 06:59:39.976098 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:39.976064 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:40.069147 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.069118 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-credential-socket\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.069147 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.069151 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5vnz\" (UniqueName: \"kubernetes.io/projected/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-kube-api-access-r5vnz\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.069385 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.069197 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-workload-socket\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.069385 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.069240 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istiod-ca-cert\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.069385 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.069291 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-data\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.069385 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.069319 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-workload-certs\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.069385 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.069338 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-envoy\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.069558 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.069416 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-podinfo\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.069558 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.069431 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-token\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.169892 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.169856 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-data\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.170074 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.169898 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-workload-certs\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.170074 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.169916 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-envoy\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.170074 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.170009 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-podinfo\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.170074 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.170043 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-token\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.170294 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.170100 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-credential-socket\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.170294 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.170127 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5vnz\" (UniqueName: \"kubernetes.io/projected/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-kube-api-access-r5vnz\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.170294 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.170240 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-data\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.170294 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.170287 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-workload-socket\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.170542 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.170321 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istiod-ca-cert\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.170542 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.170335 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-workload-certs\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.170542 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.170455 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-credential-socket\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.170741 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.170718 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-workload-socket\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.170915 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.170897 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istiod-ca-cert\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.172238 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.172219 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-envoy\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.172377 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.172344 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-podinfo\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.182884 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.182858 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5vnz\" (UniqueName: \"kubernetes.io/projected/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-kube-api-access-r5vnz\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.183980 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.183961 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-token\") pod \"e2e-mt-f08145e7-openshift-default-654764495f-bxzrp\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.253017 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.252939 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:40.396758 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.396725 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp"] Jul 07 06:59:40.400168 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.399525 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:59:40.400168 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.399647 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:59:40.400168 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.399728 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:59:40.401190 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.401162 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:40.513148 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.513054 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:40.877863 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:40.877823 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:41.059173 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:41.059137 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" event={"ID":"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a","Type":"ContainerStarted","Data":"ede738a88d7f926740ab11a9a7889b4f80a950a22e3dddb4f9052f259a1af7ca"} Jul 07 06:59:41.059354 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:41.059198 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" event={"ID":"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a","Type":"ContainerStarted","Data":"afad498222792d39df696a689c12bfcfd898318aab41f6d1e304b000651d6560"} Jul 07 06:59:41.088450 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:41.088404 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" podStartSLOduration=2.088388608 podStartE2EDuration="2.088388608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:41.087458972 +0000 UTC m=+1656.927299939" watchObservedRunningTime="2026-07-07 06:59:41.088388608 +0000 UTC m=+1656.928229572" Jul 07 06:59:41.254112 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:41.254028 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:41.259355 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:41.259331 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:42.064177 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:42.064149 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:42.065321 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:42.065299 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:43.160341 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.160307 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:43.310765 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.310711 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:43.350195 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.350162 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh"] Jul 07 06:59:43.353603 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.353584 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 06:59:43.364844 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.364824 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-f08145e7-serving-cert\"" Jul 07 06:59:43.366029 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.366016 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-754v8\"" Jul 07 06:59:43.391515 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.391488 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh"] Jul 07 06:59:43.501334 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.501233 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-openshift-service-ca\") pod \"maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh\" (UID: \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\") " pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 06:59:43.501334 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.501287 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tql6\" (UniqueName: \"kubernetes.io/projected/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-kube-api-access-2tql6\") pod \"maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh\" (UID: \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\") " pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 06:59:43.501598 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.501412 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-maas-api-tls\") pod \"maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh\" (UID: \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\") " pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 06:59:43.602854 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.602812 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-maas-api-tls\") pod \"maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh\" (UID: \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\") " pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 06:59:43.603036 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.602866 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-openshift-service-ca\") pod \"maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh\" (UID: \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\") " pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 06:59:43.603036 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.602906 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tql6\" (UniqueName: \"kubernetes.io/projected/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-kube-api-access-2tql6\") pod \"maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh\" (UID: \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\") " pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 06:59:43.603036 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:59:43.602965 2579 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-f08145e7-serving-cert: secret "maas-api-e2e-mt-f08145e7-serving-cert" not found Jul 07 06:59:43.603036 ip-10-0-130-176 kubenswrapper[2579]: E0707 06:59:43.603034 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-maas-api-tls podName:3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2 nodeName:}" failed. No retries permitted until 2026-07-07 06:59:44.103017503 +0000 UTC m=+1659.942858448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-maas-api-tls") pod "maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" (UID: "3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2") : secret "maas-api-e2e-mt-f08145e7-serving-cert" not found Jul 07 06:59:43.603538 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.603521 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-openshift-service-ca\") pod \"maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh\" (UID: \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\") " pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 06:59:43.615267 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.615237 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tql6\" (UniqueName: \"kubernetes.io/projected/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-kube-api-access-2tql6\") pod \"maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh\" (UID: \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\") " pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 06:59:43.647783 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:43.647753 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:44.068541 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:44.068505 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:44.108413 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:44.108354 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-maas-api-tls\") pod \"maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh\" (UID: \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\") " pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 06:59:44.110877 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:44.110854 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-maas-api-tls\") pod \"maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh\" (UID: \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\") " pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 06:59:44.164806 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:44.164772 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:44.263993 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:44.263949 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 06:59:44.264780 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:44.264746 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:44.407655 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:44.407624 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh"] Jul 07 06:59:44.409124 ip-10-0-130-176 kubenswrapper[2579]: W0707 06:59:44.409100 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b56c0d4_b786_4aa8_9d9e_8b86acd9d9e2.slice/crio-c3e2583fcfea7708a401a4275c1deee4268fa58968cff63b49fc076a4fd60f94 WatchSource:0}: Error finding container c3e2583fcfea7708a401a4275c1deee4268fa58968cff63b49fc076a4fd60f94: Status 404 returned error can't find the container with id c3e2583fcfea7708a401a4275c1deee4268fa58968cff63b49fc076a4fd60f94 Jul 07 06:59:44.860161 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:44.860119 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:44.969949 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:44.969909 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:45.082717 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:45.082619 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" event={"ID":"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2","Type":"ContainerStarted","Data":"ea0f8d99419a4b01b734ebedc39d0a42220b12a43e0b837c47b3185b340cdd4d"} Jul 07 06:59:45.082717 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:45.082682 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" event={"ID":"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2","Type":"ContainerStarted","Data":"c3e2583fcfea7708a401a4275c1deee4268fa58968cff63b49fc076a4fd60f94"} Jul 07 06:59:45.082914 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:45.082733 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 06:59:45.105100 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:45.105042 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" podStartSLOduration=1.713228568 podStartE2EDuration="2.105026502s" totalImagesPullingTime="391.797934ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 06:59:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 06:59:45.101960737 +0000 UTC m=+1660.941801703" watchObservedRunningTime="2026-07-07 06:59:45.105026502 +0000 UTC m=+1660.944867468" Jul 07 06:59:45.695679 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:45.695632 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:45.795737 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:45.795699 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:47.528956 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:47.528929 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:51.094894 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:51.094865 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 06:59:56.144765 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:56.144689 2579 generic.go:356] "Generic (PLEG): container finished" podID="159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" containerID="ede738a88d7f926740ab11a9a7889b4f80a950a22e3dddb4f9052f259a1af7ca" exitCode=0 Jul 07 06:59:56.145212 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:56.144754 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" event={"ID":"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a","Type":"ContainerDied","Data":"ede738a88d7f926740ab11a9a7889b4f80a950a22e3dddb4f9052f259a1af7ca"} Jul 07 06:59:56.145212 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:56.145138 2579 scope.go:117] "RemoveContainer" containerID="ede738a88d7f926740ab11a9a7889b4f80a950a22e3dddb4f9052f259a1af7ca" Jul 07 06:59:56.146143 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:56.146116 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:59:56.146258 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:56.146198 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:59:56.146258 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:56.146251 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 06:59:56.189678 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:56.189623 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 06:59:57.064787 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:57.064757 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:57.155437 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:57.155400 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" event={"ID":"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a","Type":"ContainerStarted","Data":"fd222082dbd830c93b57912e03764bc19ef298780ee96dbbc396044fcbc24a83"} Jul 07 06:59:57.253483 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:57.253444 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 06:59:57.254491 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:57.254461 2579 patch_prober.go:28] interesting pod/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 07 06:59:57.254622 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:57.254508 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" podUID="159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 07 06:59:58.254236 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:58.254199 2579 patch_prober.go:28] interesting pod/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 07 06:59:58.254677 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:58.254253 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" podUID="159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 07 06:59:59.254388 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:59.254330 2579 patch_prober.go:28] interesting pod/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 07 06:59:59.254788 ip-10-0-130-176 kubenswrapper[2579]: I0707 06:59:59.254422 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" podUID="159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 07 07:00:00.254387 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:00.254332 2579 patch_prober.go:28] interesting pod/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 07 07:00:00.254579 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:00.254410 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" podUID="159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 07 07:00:01.276676 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:01.276639 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 07:00:01.277132 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:01.276949 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 07:00:01.277800 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:01.277780 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 07:00:14.790232 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:14.790156 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:15.097145 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:15.097067 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh"] Jul 07 07:00:15.097299 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:15.097253 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" podUID="3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2" containerName="maas-api" containerID="cri-o://ea0f8d99419a4b01b734ebedc39d0a42220b12a43e0b837c47b3185b340cdd4d" gracePeriod=30 Jul 07 07:00:15.119209 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:15.119182 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:15.253272 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:15.253239 2579 generic.go:356] "Generic (PLEG): container finished" podID="3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2" containerID="ea0f8d99419a4b01b734ebedc39d0a42220b12a43e0b837c47b3185b340cdd4d" exitCode=0 Jul 07 07:00:15.253492 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:15.253289 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" event={"ID":"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2","Type":"ContainerDied","Data":"ea0f8d99419a4b01b734ebedc39d0a42220b12a43e0b837c47b3185b340cdd4d"} Jul 07 07:00:15.343476 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:15.343451 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 07:00:15.392193 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:15.392127 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tql6\" (UniqueName: \"kubernetes.io/projected/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-kube-api-access-2tql6\") pod \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\" (UID: \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\") " Jul 07 07:00:15.392322 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:15.392201 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-openshift-service-ca\") pod \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\" (UID: \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\") " Jul 07 07:00:15.392322 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:15.392298 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-maas-api-tls\") pod \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\" (UID: \"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2\") " Jul 07 07:00:15.392522 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:15.392503 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2" (UID: "3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:00:15.392625 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:15.392602 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-openshift-service-ca\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:00:15.394230 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:15.394210 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-kube-api-access-2tql6" (OuterVolumeSpecName: "kube-api-access-2tql6") pod "3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2" (UID: "3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2"). InnerVolumeSpecName "kube-api-access-2tql6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:00:15.394325 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:15.394304 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2" (UID: "3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:00:15.493621 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:15.493576 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tql6\" (UniqueName: \"kubernetes.io/projected/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-kube-api-access-2tql6\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:00:15.493621 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:15.493614 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2-maas-api-tls\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:00:16.259937 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:16.259910 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" Jul 07 07:00:16.260346 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:16.259943 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh" event={"ID":"3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2","Type":"ContainerDied","Data":"c3e2583fcfea7708a401a4275c1deee4268fa58968cff63b49fc076a4fd60f94"} Jul 07 07:00:16.260346 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:16.259995 2579 scope.go:117] "RemoveContainer" containerID="ea0f8d99419a4b01b734ebedc39d0a42220b12a43e0b837c47b3185b340cdd4d" Jul 07 07:00:16.284520 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:16.284496 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh"] Jul 07 07:00:16.288928 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:16.288896 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-f08145e7-5558c9ccbc-cddjh"] Jul 07 07:00:16.782620 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:16.782579 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2" path="/var/lib/kubelet/pods/3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2/volumes" Jul 07 07:00:20.895218 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:20.895175 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:00:25.964137 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:00:25.964098 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:02.961199 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:02.961158 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:03.033730 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:03.033695 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:03.106465 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:03.106419 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:09.326604 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:09.326526 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:09.400523 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:09.400485 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:09.589973 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:09.589894 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp"] Jul 07 07:01:09.590198 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:09.590150 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" podUID="159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" containerName="istio-proxy" containerID="cri-o://fd222082dbd830c93b57912e03764bc19ef298780ee96dbbc396044fcbc24a83" gracePeriod=30 Jul 07 07:01:09.613068 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:09.613034 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:09.844820 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:09.844734 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:10.919894 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:10.919853 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:11.023022 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:11.022989 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:11.187933 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:11.187849 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:11.928863 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:11.928825 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:12.022417 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:12.022360 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:13.539975 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:13.539945 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:14.846568 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.846544 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 07:01:14.918959 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.918890 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-envoy\") pod \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " Jul 07 07:01:14.918959 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.918922 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-podinfo\") pod \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " Jul 07 07:01:14.919135 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.918958 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-credential-socket\") pod \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " Jul 07 07:01:14.919135 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.919084 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-token\") pod \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " Jul 07 07:01:14.919210 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.919152 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5vnz\" (UniqueName: \"kubernetes.io/projected/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-kube-api-access-r5vnz\") pod \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " Jul 07 07:01:14.919305 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.919219 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istiod-ca-cert\") pod \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " Jul 07 07:01:14.919305 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.919218 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" (UID: "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:01:14.919305 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.919251 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-workload-socket\") pod \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " Jul 07 07:01:14.919305 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.919288 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-data\") pod \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " Jul 07 07:01:14.919818 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.919634 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" (UID: "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:01:14.919818 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.919760 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" (UID: "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:01:14.919818 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.919681 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-workload-certs\") pod \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\" (UID: \"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a\") " Jul 07 07:01:14.920305 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.920149 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istiod-ca-cert\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:01:14.920305 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.920172 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-workload-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:01:14.920305 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.920187 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-credential-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:01:14.920305 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.920220 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-data" (OuterVolumeSpecName: "istio-data") pod "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" (UID: "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:01:14.920305 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.920274 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" (UID: "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:01:14.921298 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.921271 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" (UID: "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:01:14.921462 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.921439 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-kube-api-access-r5vnz" (OuterVolumeSpecName: "kube-api-access-r5vnz") pod "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" (UID: "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a"). InnerVolumeSpecName "kube-api-access-r5vnz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:01:14.921551 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.921531 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-token" (OuterVolumeSpecName: "istio-token") pod "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" (UID: "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:01:14.921619 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:14.921598 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" (UID: "159bf1c6-8a2a-47e4-9b29-7fa2db246e6a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 07:01:15.021674 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.021637 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-envoy\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:01:15.021674 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.021666 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-podinfo\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:01:15.021674 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.021678 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-token\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:01:15.022170 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.021690 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5vnz\" (UniqueName: \"kubernetes.io/projected/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-kube-api-access-r5vnz\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:01:15.022170 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.021703 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-istio-data\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:01:15.022170 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.021714 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a-workload-certs\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:01:15.565996 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.565966 2579 generic.go:356] "Generic (PLEG): container finished" podID="159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" containerID="fd222082dbd830c93b57912e03764bc19ef298780ee96dbbc396044fcbc24a83" exitCode=0 Jul 07 07:01:15.566159 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.566022 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" Jul 07 07:01:15.566159 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.566053 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" event={"ID":"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a","Type":"ContainerDied","Data":"fd222082dbd830c93b57912e03764bc19ef298780ee96dbbc396044fcbc24a83"} Jul 07 07:01:15.566159 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.566113 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp" event={"ID":"159bf1c6-8a2a-47e4-9b29-7fa2db246e6a","Type":"ContainerDied","Data":"afad498222792d39df696a689c12bfcfd898318aab41f6d1e304b000651d6560"} Jul 07 07:01:15.566159 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.566134 2579 scope.go:117] "RemoveContainer" containerID="fd222082dbd830c93b57912e03764bc19ef298780ee96dbbc396044fcbc24a83" Jul 07 07:01:15.575621 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.575599 2579 scope.go:117] "RemoveContainer" containerID="ede738a88d7f926740ab11a9a7889b4f80a950a22e3dddb4f9052f259a1af7ca" Jul 07 07:01:15.583306 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.583290 2579 scope.go:117] "RemoveContainer" containerID="fd222082dbd830c93b57912e03764bc19ef298780ee96dbbc396044fcbc24a83" Jul 07 07:01:15.583558 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:01:15.583538 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd222082dbd830c93b57912e03764bc19ef298780ee96dbbc396044fcbc24a83\": container with ID starting with fd222082dbd830c93b57912e03764bc19ef298780ee96dbbc396044fcbc24a83 not found: ID does not exist" containerID="fd222082dbd830c93b57912e03764bc19ef298780ee96dbbc396044fcbc24a83" Jul 07 07:01:15.583654 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.583567 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd222082dbd830c93b57912e03764bc19ef298780ee96dbbc396044fcbc24a83"} err="failed to get container status \"fd222082dbd830c93b57912e03764bc19ef298780ee96dbbc396044fcbc24a83\": rpc error: code = NotFound desc = could not find container \"fd222082dbd830c93b57912e03764bc19ef298780ee96dbbc396044fcbc24a83\": container with ID starting with fd222082dbd830c93b57912e03764bc19ef298780ee96dbbc396044fcbc24a83 not found: ID does not exist" Jul 07 07:01:15.583654 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.583583 2579 scope.go:117] "RemoveContainer" containerID="ede738a88d7f926740ab11a9a7889b4f80a950a22e3dddb4f9052f259a1af7ca" Jul 07 07:01:15.583782 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:01:15.583766 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ede738a88d7f926740ab11a9a7889b4f80a950a22e3dddb4f9052f259a1af7ca\": container with ID starting with ede738a88d7f926740ab11a9a7889b4f80a950a22e3dddb4f9052f259a1af7ca not found: ID does not exist" containerID="ede738a88d7f926740ab11a9a7889b4f80a950a22e3dddb4f9052f259a1af7ca" Jul 07 07:01:15.583821 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.583784 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ede738a88d7f926740ab11a9a7889b4f80a950a22e3dddb4f9052f259a1af7ca"} err="failed to get container status \"ede738a88d7f926740ab11a9a7889b4f80a950a22e3dddb4f9052f259a1af7ca\": rpc error: code = NotFound desc = could not find container \"ede738a88d7f926740ab11a9a7889b4f80a950a22e3dddb4f9052f259a1af7ca\": container with ID starting with ede738a88d7f926740ab11a9a7889b4f80a950a22e3dddb4f9052f259a1af7ca not found: ID does not exist" Jul 07 07:01:15.590072 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.590051 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp"] Jul 07 07:01:15.594793 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:15.594774 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f08145e7-openshift-default-654764495f-bxzrp"] Jul 07 07:01:16.782446 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:16.782411 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" path="/var/lib/kubelet/pods/159bf1c6-8a2a-47e4-9b29-7fa2db246e6a/volumes" Jul 07 07:01:19.560536 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:19.560500 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:19.660310 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:19.660274 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:20.265053 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:20.265020 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:26.648095 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:26.648061 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:26.749432 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:26.749401 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:36.816956 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:36.816916 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:37.175432 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:37.175316 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:01:58.213973 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:01:58.213884 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:04.544949 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:04.544915 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:04.626025 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:04.625994 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:04.811701 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:04.811630 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 07:02:04.816018 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:04.815996 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 07:02:04.908503 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:04.908463 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:05.201720 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.201639 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n"] Jul 07 07:02:05.202081 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.202064 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2" containerName="maas-api" Jul 07 07:02:05.202132 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.202085 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2" containerName="maas-api" Jul 07 07:02:05.202132 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.202116 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" containerName="istio-proxy" Jul 07 07:02:05.202132 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.202125 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" containerName="istio-proxy" Jul 07 07:02:05.202240 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.202138 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" containerName="istio-proxy" Jul 07 07:02:05.202240 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.202147 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" containerName="istio-proxy" Jul 07 07:02:05.202240 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.202233 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" containerName="istio-proxy" Jul 07 07:02:05.202337 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.202244 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b56c0d4-b786-4aa8-9d9e-8b86acd9d9e2" containerName="maas-api" Jul 07 07:02:05.206320 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.206301 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.209541 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.209524 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-9b5a3c-openshift-default-dockercfg-4s74f\"" Jul 07 07:02:05.219013 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.218993 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n"] Jul 07 07:02:05.229780 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.229753 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:05.349616 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.349582 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:05.362266 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.362238 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-istio-data\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.362416 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.362280 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-workload-socket\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.362416 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.362309 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-workload-certs\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.362540 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.362440 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-istio-envoy\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.362540 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.362469 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a94d5c-baab-4af0-b012-e7a38b237370-istiod-ca-cert\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.362540 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.362498 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-credential-socket\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.362540 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.362531 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a94d5c-baab-4af0-b012-e7a38b237370-istio-podinfo\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.362667 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.362550 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a94d5c-baab-4af0-b012-e7a38b237370-istio-token\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.362667 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.362570 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzqch\" (UniqueName: \"kubernetes.io/projected/92a94d5c-baab-4af0-b012-e7a38b237370-kube-api-access-pzqch\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.463749 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.463651 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-workload-certs\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.463749 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.463737 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-istio-envoy\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.463976 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.463757 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a94d5c-baab-4af0-b012-e7a38b237370-istiod-ca-cert\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.463976 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.463782 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-credential-socket\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.463976 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.463802 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a94d5c-baab-4af0-b012-e7a38b237370-istio-podinfo\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.463976 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.463817 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a94d5c-baab-4af0-b012-e7a38b237370-istio-token\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.463976 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.463838 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzqch\" (UniqueName: \"kubernetes.io/projected/92a94d5c-baab-4af0-b012-e7a38b237370-kube-api-access-pzqch\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.463976 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.463886 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-istio-data\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.463976 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.463932 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-workload-socket\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.464317 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.464178 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-credential-socket\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.464317 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.464216 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-workload-certs\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.464317 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.464300 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-workload-socket\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.464478 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.464399 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-istio-data\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.464544 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.464525 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a94d5c-baab-4af0-b012-e7a38b237370-istiod-ca-cert\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.466221 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.466202 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-istio-envoy\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.466314 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.466296 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a94d5c-baab-4af0-b012-e7a38b237370-istio-podinfo\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.473631 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.473613 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a94d5c-baab-4af0-b012-e7a38b237370-istio-token\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.473718 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.473659 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzqch\" (UniqueName: \"kubernetes.io/projected/92a94d5c-baab-4af0-b012-e7a38b237370-kube-api-access-pzqch\") pod \"e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.516285 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.516261 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:05.648200 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.648175 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n"] Jul 07 07:02:05.650484 ip-10-0-130-176 kubenswrapper[2579]: W0707 07:02:05.650459 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92a94d5c_baab_4af0_b012_e7a38b237370.slice/crio-511b004f3c319ad862c1400e66c8a0456db3080d2489fab85fd6fed20d63735a WatchSource:0}: Error finding container 511b004f3c319ad862c1400e66c8a0456db3080d2489fab85fd6fed20d63735a: Status 404 returned error can't find the container with id 511b004f3c319ad862c1400e66c8a0456db3080d2489fab85fd6fed20d63735a Jul 07 07:02:05.652854 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.652825 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:02:05.652937 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.652889 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:02:05.652937 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.652915 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:02:05.825413 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.825382 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" event={"ID":"92a94d5c-baab-4af0-b012-e7a38b237370","Type":"ContainerStarted","Data":"fd3460894dd97c70a4ff6b852804c9bee7bc84699ad91b8075e391e5926f1aaa"} Jul 07 07:02:05.825597 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.825434 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" event={"ID":"92a94d5c-baab-4af0-b012-e7a38b237370","Type":"ContainerStarted","Data":"511b004f3c319ad862c1400e66c8a0456db3080d2489fab85fd6fed20d63735a"} Jul 07 07:02:05.852321 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:05.852261 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" podStartSLOduration=0.852238432 podStartE2EDuration="852.238432ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:02:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:02:05.849882229 +0000 UTC m=+1801.689723199" watchObservedRunningTime="2026-07-07 07:02:05.852238432 +0000 UTC m=+1801.692079401" Jul 07 07:02:06.517307 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:06.517278 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:06.521876 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:06.521855 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:06.830034 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:06.830006 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:06.831053 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:06.831034 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:07.847830 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:07.847778 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:07.935027 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:07.934992 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9"] Jul 07 07:02:07.935519 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:07.935488 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="159bf1c6-8a2a-47e4-9b29-7fa2db246e6a" containerName="istio-proxy" Jul 07 07:02:07.937390 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:07.937375 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" Jul 07 07:02:07.940224 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:07.940204 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-9b5a3c-serving-cert\"" Jul 07 07:02:07.940329 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:07.940302 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-754v8\"" Jul 07 07:02:07.948865 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:07.948837 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:07.949662 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:07.949638 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9"] Jul 07 07:02:08.059660 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.059629 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:08.088248 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.088213 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9\" (UID: \"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3\") " pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" Jul 07 07:02:08.088481 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.088336 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9jzv\" (UniqueName: \"kubernetes.io/projected/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-kube-api-access-c9jzv\") pod \"maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9\" (UID: \"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3\") " pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" Jul 07 07:02:08.088481 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.088465 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-maas-api-tls\") pod \"maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9\" (UID: \"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3\") " pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" Jul 07 07:02:08.157091 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.157001 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:08.189404 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.189376 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-maas-api-tls\") pod \"maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9\" (UID: \"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3\") " pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" Jul 07 07:02:08.189579 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.189409 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9\" (UID: \"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3\") " pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" Jul 07 07:02:08.189579 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.189464 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9jzv\" (UniqueName: \"kubernetes.io/projected/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-kube-api-access-c9jzv\") pod \"maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9\" (UID: \"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3\") " pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" Jul 07 07:02:08.190122 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.190103 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9\" (UID: \"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3\") " pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" Jul 07 07:02:08.191898 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.191870 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-maas-api-tls\") pod \"maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9\" (UID: \"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3\") " pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" Jul 07 07:02:08.204316 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.204289 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9jzv\" (UniqueName: \"kubernetes.io/projected/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-kube-api-access-c9jzv\") pod \"maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9\" (UID: \"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3\") " pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" Jul 07 07:02:08.249094 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.249067 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" Jul 07 07:02:08.402976 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.402944 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9"] Jul 07 07:02:08.405981 ip-10-0-130-176 kubenswrapper[2579]: W0707 07:02:08.405953 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6fdedf8_15fd_4d5c_b010_f63d1a8387a3.slice/crio-166392458b6fabe6c963e27ba6a0fc92a72e0a36ba5ada669e58102baca2284e WatchSource:0}: Error finding container 166392458b6fabe6c963e27ba6a0fc92a72e0a36ba5ada669e58102baca2284e: Status 404 returned error can't find the container with id 166392458b6fabe6c963e27ba6a0fc92a72e0a36ba5ada669e58102baca2284e Jul 07 07:02:08.407355 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.407307 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 07:02:08.497625 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.497589 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:08.605972 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.605941 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:08.842230 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:08.842195 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" event={"ID":"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3","Type":"ContainerStarted","Data":"166392458b6fabe6c963e27ba6a0fc92a72e0a36ba5ada669e58102baca2284e"} Jul 07 07:02:09.848312 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:09.848279 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" event={"ID":"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3","Type":"ContainerStarted","Data":"d3a6c3d51c0a09c1dea29ae88473449d7311dea15d84dea504ecd7c4b0554737"} Jul 07 07:02:09.848749 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:09.848408 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" Jul 07 07:02:09.867027 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:09.866982 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" podStartSLOduration=2.405062283 podStartE2EDuration="2.866968211s" totalImagesPullingTime="461.905928ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:02:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:02:09.865954545 +0000 UTC m=+1805.705795523" watchObservedRunningTime="2026-07-07 07:02:09.866968211 +0000 UTC m=+1805.706809234" Jul 07 07:02:12.378798 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:12.378757 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:12.719239 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:12.719205 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:12.997878 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:12.997801 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r"] Jul 07 07:02:13.000391 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.000350 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.004132 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.004111 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-da1ed1-openshift-default-dockercfg-24szg\"" Jul 07 07:02:13.015623 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.015602 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r"] Jul 07 07:02:13.026164 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.026137 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:13.136184 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.136150 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-credential-socket\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.136355 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.136192 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-envoy\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.136355 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.136216 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m87xq\" (UniqueName: \"kubernetes.io/projected/755b78f5-0288-4249-bcfe-bb5283c43cb6-kube-api-access-m87xq\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.136355 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.136299 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-data\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.136603 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.136392 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-workload-socket\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.136603 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.136461 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-token\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.136603 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.136547 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/755b78f5-0288-4249-bcfe-bb5283c43cb6-istiod-ca-cert\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.136603 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.136597 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-podinfo\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.136766 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.136627 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-workload-certs\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.237992 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.237962 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-podinfo\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.237992 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.237993 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-workload-certs\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.238236 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.238018 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-credential-socket\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.238236 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.238052 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-envoy\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.238236 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.238082 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m87xq\" (UniqueName: \"kubernetes.io/projected/755b78f5-0288-4249-bcfe-bb5283c43cb6-kube-api-access-m87xq\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.238236 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.238117 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-data\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.238236 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.238150 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-workload-socket\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.238236 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.238198 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-token\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.238576 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.238262 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/755b78f5-0288-4249-bcfe-bb5283c43cb6-istiod-ca-cert\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.238576 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.238481 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-workload-certs\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.238673 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.238589 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-credential-socket\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.238764 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.238719 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-data\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.238859 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.238829 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-workload-socket\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.239038 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.239021 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/755b78f5-0288-4249-bcfe-bb5283c43cb6-istiod-ca-cert\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.240426 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.240405 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-envoy\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.240667 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.240651 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-podinfo\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.247734 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.247710 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m87xq\" (UniqueName: \"kubernetes.io/projected/755b78f5-0288-4249-bcfe-bb5283c43cb6-kube-api-access-m87xq\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.248198 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.248146 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-token\") pod \"e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.310884 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.310858 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:13.448461 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.448434 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r"] Jul 07 07:02:13.449865 ip-10-0-130-176 kubenswrapper[2579]: W0707 07:02:13.449837 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod755b78f5_0288_4249_bcfe_bb5283c43cb6.slice/crio-5aa8f1a4b323dfc1aea3b9bdff7c8d7f8c013b8c85c60b86c2878ad6c2b9cf25 WatchSource:0}: Error finding container 5aa8f1a4b323dfc1aea3b9bdff7c8d7f8c013b8c85c60b86c2878ad6c2b9cf25: Status 404 returned error can't find the container with id 5aa8f1a4b323dfc1aea3b9bdff7c8d7f8c013b8c85c60b86c2878ad6c2b9cf25 Jul 07 07:02:13.452427 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.452394 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:02:13.452535 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.452455 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:02:13.452535 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.452490 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:02:13.868906 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.868873 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" event={"ID":"755b78f5-0288-4249-bcfe-bb5283c43cb6","Type":"ContainerStarted","Data":"d6fc4f33b08e1eb5746d198ee39dbfa3dbbf160105ff6894195382fcc64782c5"} Jul 07 07:02:13.869121 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.868922 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" event={"ID":"755b78f5-0288-4249-bcfe-bb5283c43cb6","Type":"ContainerStarted","Data":"5aa8f1a4b323dfc1aea3b9bdff7c8d7f8c013b8c85c60b86c2878ad6c2b9cf25"} Jul 07 07:02:13.894152 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:13.894097 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" podStartSLOduration=1.8940829030000002 podStartE2EDuration="1.894082903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:02:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:02:13.892502971 +0000 UTC m=+1809.732343952" watchObservedRunningTime="2026-07-07 07:02:13.894082903 +0000 UTC m=+1809.733923871" Jul 07 07:02:14.086795 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:14.086764 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:14.311411 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:14.311346 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:14.316733 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:14.316709 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:14.875145 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:14.875108 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:14.876153 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:14.876130 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:15.862400 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:15.862306 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" Jul 07 07:02:15.871819 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:15.871786 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:15.963919 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:15.963787 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9"] Jul 07 07:02:15.967383 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:15.967337 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:02:15.970234 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:15.970210 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-da1ed1-serving-cert\"" Jul 07 07:02:15.970818 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:15.970789 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:15.983703 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:15.983679 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9"] Jul 07 07:02:16.069069 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:16.069028 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322fc4ff-e052-4e6f-b513-274dddec2e70-maas-api-tls\") pod \"maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9\" (UID: \"322fc4ff-e052-4e6f-b513-274dddec2e70\") " pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:02:16.069258 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:16.069077 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8gs2\" (UniqueName: \"kubernetes.io/projected/322fc4ff-e052-4e6f-b513-274dddec2e70-kube-api-access-z8gs2\") pod \"maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9\" (UID: \"322fc4ff-e052-4e6f-b513-274dddec2e70\") " pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:02:16.069258 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:16.069156 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/322fc4ff-e052-4e6f-b513-274dddec2e70-openshift-service-ca\") pod \"maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9\" (UID: \"322fc4ff-e052-4e6f-b513-274dddec2e70\") " pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:02:16.123224 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:16.123133 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:16.170693 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:16.170660 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322fc4ff-e052-4e6f-b513-274dddec2e70-maas-api-tls\") pod \"maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9\" (UID: \"322fc4ff-e052-4e6f-b513-274dddec2e70\") " pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:02:16.170693 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:16.170694 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8gs2\" (UniqueName: \"kubernetes.io/projected/322fc4ff-e052-4e6f-b513-274dddec2e70-kube-api-access-z8gs2\") pod \"maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9\" (UID: \"322fc4ff-e052-4e6f-b513-274dddec2e70\") " pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:02:16.170895 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:02:16.170803 2579 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-da1ed1-serving-cert: secret "maas-api-e2e-inf-b-da1ed1-serving-cert" not found Jul 07 07:02:16.170895 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:02:16.170861 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/322fc4ff-e052-4e6f-b513-274dddec2e70-maas-api-tls podName:322fc4ff-e052-4e6f-b513-274dddec2e70 nodeName:}" failed. No retries permitted until 2026-07-07 07:02:16.670844118 +0000 UTC m=+1812.510685063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/322fc4ff-e052-4e6f-b513-274dddec2e70-maas-api-tls") pod "maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" (UID: "322fc4ff-e052-4e6f-b513-274dddec2e70") : secret "maas-api-e2e-inf-b-da1ed1-serving-cert" not found Jul 07 07:02:16.170895 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:16.170883 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/322fc4ff-e052-4e6f-b513-274dddec2e70-openshift-service-ca\") pod \"maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9\" (UID: \"322fc4ff-e052-4e6f-b513-274dddec2e70\") " pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:02:16.171462 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:16.171446 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/322fc4ff-e052-4e6f-b513-274dddec2e70-openshift-service-ca\") pod \"maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9\" (UID: \"322fc4ff-e052-4e6f-b513-274dddec2e70\") " pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:02:16.190616 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:16.190582 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8gs2\" (UniqueName: \"kubernetes.io/projected/322fc4ff-e052-4e6f-b513-274dddec2e70-kube-api-access-z8gs2\") pod \"maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9\" (UID: \"322fc4ff-e052-4e6f-b513-274dddec2e70\") " pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:02:16.308429 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:16.308393 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:16.556118 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:16.556079 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:16.655092 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:16.655055 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:16.676342 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:16.676309 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322fc4ff-e052-4e6f-b513-274dddec2e70-maas-api-tls\") pod \"maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9\" (UID: \"322fc4ff-e052-4e6f-b513-274dddec2e70\") " pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:02:16.678743 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:16.678727 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322fc4ff-e052-4e6f-b513-274dddec2e70-maas-api-tls\") pod \"maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9\" (UID: \"322fc4ff-e052-4e6f-b513-274dddec2e70\") " pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:02:16.880665 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:16.880575 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:02:17.083108 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:17.083086 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9"] Jul 07 07:02:17.084959 ip-10-0-130-176 kubenswrapper[2579]: W0707 07:02:17.084927 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod322fc4ff_e052_4e6f_b513_274dddec2e70.slice/crio-914f59c92ed941d81ad34eb2338c1d1689ae7df5f8c1dc623c6d05fad433a467 WatchSource:0}: Error finding container 914f59c92ed941d81ad34eb2338c1d1689ae7df5f8c1dc623c6d05fad433a467: Status 404 returned error can't find the container with id 914f59c92ed941d81ad34eb2338c1d1689ae7df5f8c1dc623c6d05fad433a467 Jul 07 07:02:17.891404 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:17.891291 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" event={"ID":"322fc4ff-e052-4e6f-b513-274dddec2e70","Type":"ContainerStarted","Data":"01d76a161868364a485bea2055f3130af5f2b78b894bbd5bf5c8236252f970fe"} Jul 07 07:02:17.891404 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:17.891354 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" event={"ID":"322fc4ff-e052-4e6f-b513-274dddec2e70","Type":"ContainerStarted","Data":"914f59c92ed941d81ad34eb2338c1d1689ae7df5f8c1dc623c6d05fad433a467"} Jul 07 07:02:17.891611 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:17.891406 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:02:20.473649 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:20.473611 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:22.065305 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:22.065273 2579 patch_prober.go:28] interesting pod/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": read tcp 10.134.0.2:50144->10.134.0.44:15021: read: connection reset by peer" start-of-body= Jul 07 07:02:22.065698 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:22.065337 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" podUID="92a94d5c-baab-4af0-b012-e7a38b237370" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": read tcp 10.134.0.2:50144->10.134.0.44:15021: read: connection reset by peer" Jul 07 07:02:22.919660 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:22.919629 2579 generic.go:356] "Generic (PLEG): container finished" podID="92a94d5c-baab-4af0-b012-e7a38b237370" containerID="fd3460894dd97c70a4ff6b852804c9bee7bc84699ad91b8075e391e5926f1aaa" exitCode=0 Jul 07 07:02:22.919847 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:22.919673 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" event={"ID":"92a94d5c-baab-4af0-b012-e7a38b237370","Type":"ContainerDied","Data":"fd3460894dd97c70a4ff6b852804c9bee7bc84699ad91b8075e391e5926f1aaa"} Jul 07 07:02:22.920059 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:22.920041 2579 scope.go:117] "RemoveContainer" containerID="fd3460894dd97c70a4ff6b852804c9bee7bc84699ad91b8075e391e5926f1aaa" Jul 07 07:02:22.921015 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:22.920990 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:02:22.921067 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:22.921052 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:02:22.921105 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:22.921084 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:02:22.955386 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:22.954341 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" podStartSLOduration=7.383238452 podStartE2EDuration="7.954320008s" totalImagesPullingTime="571.081556ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:02:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:02:17.916985884 +0000 UTC m=+1813.756826852" watchObservedRunningTime="2026-07-07 07:02:22.954320008 +0000 UTC m=+1818.794160977" Jul 07 07:02:23.902757 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:23.902731 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:02:23.927602 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:23.927571 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" event={"ID":"92a94d5c-baab-4af0-b012-e7a38b237370","Type":"ContainerStarted","Data":"611c889b321874a73c5cffd0eece245d10fdd4994798f4f22d9c9905d47c6279"} Jul 07 07:02:24.517449 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:24.517408 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:24.517695 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:24.517670 2579 patch_prober.go:28] interesting pod/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 07 07:02:24.517782 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:24.517728 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" podUID="92a94d5c-baab-4af0-b012-e7a38b237370" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 07 07:02:25.517595 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:25.517547 2579 patch_prober.go:28] interesting pod/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 07 07:02:25.518004 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:25.517640 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" podUID="92a94d5c-baab-4af0-b012-e7a38b237370" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 07 07:02:26.517496 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:26.517463 2579 patch_prober.go:28] interesting pod/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 07 07:02:26.517665 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:26.517520 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" podUID="92a94d5c-baab-4af0-b012-e7a38b237370" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 07 07:02:27.517098 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:27.517017 2579 patch_prober.go:28] interesting pod/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 07 07:02:27.517098 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:27.517073 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" podUID="92a94d5c-baab-4af0-b012-e7a38b237370" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 07 07:02:27.965047 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:27.965015 2579 generic.go:356] "Generic (PLEG): container finished" podID="755b78f5-0288-4249-bcfe-bb5283c43cb6" containerID="d6fc4f33b08e1eb5746d198ee39dbfa3dbbf160105ff6894195382fcc64782c5" exitCode=0 Jul 07 07:02:27.965467 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:27.965088 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" event={"ID":"755b78f5-0288-4249-bcfe-bb5283c43cb6","Type":"ContainerDied","Data":"d6fc4f33b08e1eb5746d198ee39dbfa3dbbf160105ff6894195382fcc64782c5"} Jul 07 07:02:27.965582 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:27.965568 2579 scope.go:117] "RemoveContainer" containerID="d6fc4f33b08e1eb5746d198ee39dbfa3dbbf160105ff6894195382fcc64782c5" Jul 07 07:02:27.966587 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:27.966559 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:02:27.966703 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:27.966640 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:02:27.966703 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:27.966687 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:02:28.520279 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:28.520253 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:28.520616 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:28.520595 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:28.521306 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:28.521286 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:02:28.973115 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:28.973080 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" event={"ID":"755b78f5-0288-4249-bcfe-bb5283c43cb6","Type":"ContainerStarted","Data":"8c1b35809b1f65ac23921cfe80dc90ea2a70da5036e0385ae3428e86519bf49c"} Jul 07 07:02:29.311579 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:29.311536 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:29.312739 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:29.312713 2579 patch_prober.go:28] interesting pod/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 07 07:02:29.312848 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:29.312763 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" podUID="755b78f5-0288-4249-bcfe-bb5283c43cb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 07 07:02:29.338674 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:29.338634 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:29.433984 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:29.433939 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:29.875478 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:29.875438 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:30.311229 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:30.311194 2579 patch_prober.go:28] interesting pod/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 07 07:02:30.311639 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:30.311262 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" podUID="755b78f5-0288-4249-bcfe-bb5283c43cb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 07 07:02:31.311846 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:31.311802 2579 patch_prober.go:28] interesting pod/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 07 07:02:31.312321 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:31.311880 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" podUID="755b78f5-0288-4249-bcfe-bb5283c43cb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 07 07:02:32.312044 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:32.312008 2579 patch_prober.go:28] interesting pod/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 07 07:02:32.312452 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:32.312076 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" podUID="755b78f5-0288-4249-bcfe-bb5283c43cb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 07 07:02:33.315107 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:33.315072 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:33.316050 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:33.316033 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:02:37.508150 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:37.508098 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:37.606623 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:37.606574 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:02:43.058352 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:02:43.058313 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:03:14.072277 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:14.072224 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:03:39.696329 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:39.696238 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:03:40.615575 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:40.615534 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:03:40.749904 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:40.749862 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:03:41.147546 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.147499 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:03:41.274112 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.274084 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:03:41.373623 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.373584 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:03:41.521670 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.521594 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9"] Jul 07 07:03:41.521838 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.521789 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" podUID="d6fdedf8-15fd-4d5c-b010-f63d1a8387a3" containerName="maas-api" containerID="cri-o://d3a6c3d51c0a09c1dea29ae88473449d7311dea15d84dea504ecd7c4b0554737" gracePeriod=30 Jul 07 07:03:41.565764 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.565733 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:03:41.763471 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.763447 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" Jul 07 07:03:41.898570 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.898536 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9jzv\" (UniqueName: \"kubernetes.io/projected/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-kube-api-access-c9jzv\") pod \"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3\" (UID: \"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3\") " Jul 07 07:03:41.898747 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.898580 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-openshift-service-ca\") pod \"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3\" (UID: \"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3\") " Jul 07 07:03:41.898747 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.898685 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-maas-api-tls\") pod \"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3\" (UID: \"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3\") " Jul 07 07:03:41.898983 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.898959 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d6fdedf8-15fd-4d5c-b010-f63d1a8387a3" (UID: "d6fdedf8-15fd-4d5c-b010-f63d1a8387a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:03:41.900739 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.900711 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-kube-api-access-c9jzv" (OuterVolumeSpecName: "kube-api-access-c9jzv") pod "d6fdedf8-15fd-4d5c-b010-f63d1a8387a3" (UID: "d6fdedf8-15fd-4d5c-b010-f63d1a8387a3"). InnerVolumeSpecName "kube-api-access-c9jzv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:03:41.900864 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.900739 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d6fdedf8-15fd-4d5c-b010-f63d1a8387a3" (UID: "d6fdedf8-15fd-4d5c-b010-f63d1a8387a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:03:41.999321 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.999284 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-maas-api-tls\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:03:41.999321 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.999314 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9jzv\" (UniqueName: \"kubernetes.io/projected/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-kube-api-access-c9jzv\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:03:41.999536 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:41.999330 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3-openshift-service-ca\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:03:42.371308 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:42.371270 2579 generic.go:356] "Generic (PLEG): container finished" podID="d6fdedf8-15fd-4d5c-b010-f63d1a8387a3" containerID="d3a6c3d51c0a09c1dea29ae88473449d7311dea15d84dea504ecd7c4b0554737" exitCode=0 Jul 07 07:03:42.371493 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:42.371318 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" Jul 07 07:03:42.371493 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:42.371342 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" event={"ID":"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3","Type":"ContainerDied","Data":"d3a6c3d51c0a09c1dea29ae88473449d7311dea15d84dea504ecd7c4b0554737"} Jul 07 07:03:42.371493 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:42.371411 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9" event={"ID":"d6fdedf8-15fd-4d5c-b010-f63d1a8387a3","Type":"ContainerDied","Data":"166392458b6fabe6c963e27ba6a0fc92a72e0a36ba5ada669e58102baca2284e"} Jul 07 07:03:42.371493 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:42.371441 2579 scope.go:117] "RemoveContainer" containerID="d3a6c3d51c0a09c1dea29ae88473449d7311dea15d84dea504ecd7c4b0554737" Jul 07 07:03:42.380869 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:42.380850 2579 scope.go:117] "RemoveContainer" containerID="d3a6c3d51c0a09c1dea29ae88473449d7311dea15d84dea504ecd7c4b0554737" Jul 07 07:03:42.381112 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:03:42.381090 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3a6c3d51c0a09c1dea29ae88473449d7311dea15d84dea504ecd7c4b0554737\": container with ID starting with d3a6c3d51c0a09c1dea29ae88473449d7311dea15d84dea504ecd7c4b0554737 not found: ID does not exist" containerID="d3a6c3d51c0a09c1dea29ae88473449d7311dea15d84dea504ecd7c4b0554737" Jul 07 07:03:42.381168 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:42.381121 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a6c3d51c0a09c1dea29ae88473449d7311dea15d84dea504ecd7c4b0554737"} err="failed to get container status \"d3a6c3d51c0a09c1dea29ae88473449d7311dea15d84dea504ecd7c4b0554737\": rpc error: code = NotFound desc = could not find container \"d3a6c3d51c0a09c1dea29ae88473449d7311dea15d84dea504ecd7c4b0554737\": container with ID starting with d3a6c3d51c0a09c1dea29ae88473449d7311dea15d84dea504ecd7c4b0554737 not found: ID does not exist" Jul 07 07:03:42.398170 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:42.398142 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9"] Jul 07 07:03:42.403101 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:42.403082 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-9b5a3c-5fb86695f4-nd4b9"] Jul 07 07:03:42.783504 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:42.783472 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6fdedf8-15fd-4d5c-b010-f63d1a8387a3" path="/var/lib/kubelet/pods/d6fdedf8-15fd-4d5c-b010-f63d1a8387a3/volumes" Jul 07 07:03:46.222640 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:03:46.222608 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:08.986647 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:08.986616 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:09.087221 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:09.087193 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:13.530999 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:13.530963 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:22.973808 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:22.973770 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:23.227886 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:23.227796 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n"] Jul 07 07:04:23.228042 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:23.228007 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" podUID="92a94d5c-baab-4af0-b012-e7a38b237370" containerName="istio-proxy" containerID="cri-o://611c889b321874a73c5cffd0eece245d10fdd4994798f4f22d9c9905d47c6279" gracePeriod=30 Jul 07 07:04:23.250556 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:23.250524 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:23.468981 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:23.468937 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:23.623940 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:23.623906 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:24.019346 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.019259 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:24.147690 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.147650 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:24.293220 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.293190 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9"] Jul 07 07:04:24.293451 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.293416 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" podUID="322fc4ff-e052-4e6f-b513-274dddec2e70" containerName="maas-api" containerID="cri-o://01d76a161868364a485bea2055f3130af5f2b78b894bbd5bf5c8236252f970fe" gracePeriod=30 Jul 07 07:04:24.327994 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.327963 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:24.535894 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.535869 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:04:24.596387 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.596305 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8gs2\" (UniqueName: \"kubernetes.io/projected/322fc4ff-e052-4e6f-b513-274dddec2e70-kube-api-access-z8gs2\") pod \"322fc4ff-e052-4e6f-b513-274dddec2e70\" (UID: \"322fc4ff-e052-4e6f-b513-274dddec2e70\") " Jul 07 07:04:24.596387 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.596344 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/322fc4ff-e052-4e6f-b513-274dddec2e70-openshift-service-ca\") pod \"322fc4ff-e052-4e6f-b513-274dddec2e70\" (UID: \"322fc4ff-e052-4e6f-b513-274dddec2e70\") " Jul 07 07:04:24.596582 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.596452 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322fc4ff-e052-4e6f-b513-274dddec2e70-maas-api-tls\") pod \"322fc4ff-e052-4e6f-b513-274dddec2e70\" (UID: \"322fc4ff-e052-4e6f-b513-274dddec2e70\") " Jul 07 07:04:24.596728 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.596707 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/322fc4ff-e052-4e6f-b513-274dddec2e70-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "322fc4ff-e052-4e6f-b513-274dddec2e70" (UID: "322fc4ff-e052-4e6f-b513-274dddec2e70"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:04:24.596861 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.596843 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/322fc4ff-e052-4e6f-b513-274dddec2e70-openshift-service-ca\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:24.598389 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.598356 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/322fc4ff-e052-4e6f-b513-274dddec2e70-kube-api-access-z8gs2" (OuterVolumeSpecName: "kube-api-access-z8gs2") pod "322fc4ff-e052-4e6f-b513-274dddec2e70" (UID: "322fc4ff-e052-4e6f-b513-274dddec2e70"). InnerVolumeSpecName "kube-api-access-z8gs2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:04:24.598495 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.598470 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/322fc4ff-e052-4e6f-b513-274dddec2e70-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "322fc4ff-e052-4e6f-b513-274dddec2e70" (UID: "322fc4ff-e052-4e6f-b513-274dddec2e70"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:04:24.606254 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.606227 2579 generic.go:356] "Generic (PLEG): container finished" podID="322fc4ff-e052-4e6f-b513-274dddec2e70" containerID="01d76a161868364a485bea2055f3130af5f2b78b894bbd5bf5c8236252f970fe" exitCode=0 Jul 07 07:04:24.606384 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.606279 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" Jul 07 07:04:24.606384 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.606316 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" event={"ID":"322fc4ff-e052-4e6f-b513-274dddec2e70","Type":"ContainerDied","Data":"01d76a161868364a485bea2055f3130af5f2b78b894bbd5bf5c8236252f970fe"} Jul 07 07:04:24.606474 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.606384 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9" event={"ID":"322fc4ff-e052-4e6f-b513-274dddec2e70","Type":"ContainerDied","Data":"914f59c92ed941d81ad34eb2338c1d1689ae7df5f8c1dc623c6d05fad433a467"} Jul 07 07:04:24.606474 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.606409 2579 scope.go:117] "RemoveContainer" containerID="01d76a161868364a485bea2055f3130af5f2b78b894bbd5bf5c8236252f970fe" Jul 07 07:04:24.617543 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.617525 2579 scope.go:117] "RemoveContainer" containerID="01d76a161868364a485bea2055f3130af5f2b78b894bbd5bf5c8236252f970fe" Jul 07 07:04:24.617811 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:04:24.617790 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01d76a161868364a485bea2055f3130af5f2b78b894bbd5bf5c8236252f970fe\": container with ID starting with 01d76a161868364a485bea2055f3130af5f2b78b894bbd5bf5c8236252f970fe not found: ID does not exist" containerID="01d76a161868364a485bea2055f3130af5f2b78b894bbd5bf5c8236252f970fe" Jul 07 07:04:24.617881 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.617817 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01d76a161868364a485bea2055f3130af5f2b78b894bbd5bf5c8236252f970fe"} err="failed to get container status \"01d76a161868364a485bea2055f3130af5f2b78b894bbd5bf5c8236252f970fe\": rpc error: code = NotFound desc = could not find container \"01d76a161868364a485bea2055f3130af5f2b78b894bbd5bf5c8236252f970fe\": container with ID starting with 01d76a161868364a485bea2055f3130af5f2b78b894bbd5bf5c8236252f970fe not found: ID does not exist" Jul 07 07:04:24.628482 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.628455 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9"] Jul 07 07:04:24.634840 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.634818 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-da1ed1-788b4c4d4-bmcr9"] Jul 07 07:04:24.698107 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.698074 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8gs2\" (UniqueName: \"kubernetes.io/projected/322fc4ff-e052-4e6f-b513-274dddec2e70-kube-api-access-z8gs2\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:24.698107 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.698101 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322fc4ff-e052-4e6f-b513-274dddec2e70-maas-api-tls\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:24.782743 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:24.782710 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="322fc4ff-e052-4e6f-b513-274dddec2e70" path="/var/lib/kubelet/pods/322fc4ff-e052-4e6f-b513-274dddec2e70/volumes" Jul 07 07:04:25.387230 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:25.387199 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:25.488381 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:25.488333 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:28.481969 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.481947 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:04:28.629026 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.628991 2579 generic.go:356] "Generic (PLEG): container finished" podID="92a94d5c-baab-4af0-b012-e7a38b237370" containerID="611c889b321874a73c5cffd0eece245d10fdd4994798f4f22d9c9905d47c6279" exitCode=0 Jul 07 07:04:28.629218 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.629038 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" Jul 07 07:04:28.629218 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.629076 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" event={"ID":"92a94d5c-baab-4af0-b012-e7a38b237370","Type":"ContainerDied","Data":"611c889b321874a73c5cffd0eece245d10fdd4994798f4f22d9c9905d47c6279"} Jul 07 07:04:28.629218 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.629129 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n" event={"ID":"92a94d5c-baab-4af0-b012-e7a38b237370","Type":"ContainerDied","Data":"511b004f3c319ad862c1400e66c8a0456db3080d2489fab85fd6fed20d63735a"} Jul 07 07:04:28.629218 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.629153 2579 scope.go:117] "RemoveContainer" containerID="611c889b321874a73c5cffd0eece245d10fdd4994798f4f22d9c9905d47c6279" Jul 07 07:04:28.635177 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.635157 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a94d5c-baab-4af0-b012-e7a38b237370-istio-podinfo\") pod \"92a94d5c-baab-4af0-b012-e7a38b237370\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " Jul 07 07:04:28.635284 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.635187 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-istio-envoy\") pod \"92a94d5c-baab-4af0-b012-e7a38b237370\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " Jul 07 07:04:28.635284 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.635229 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-workload-socket\") pod \"92a94d5c-baab-4af0-b012-e7a38b237370\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " Jul 07 07:04:28.635416 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.635343 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzqch\" (UniqueName: \"kubernetes.io/projected/92a94d5c-baab-4af0-b012-e7a38b237370-kube-api-access-pzqch\") pod \"92a94d5c-baab-4af0-b012-e7a38b237370\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " Jul 07 07:04:28.635416 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.635408 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-istio-data\") pod \"92a94d5c-baab-4af0-b012-e7a38b237370\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " Jul 07 07:04:28.635527 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.635436 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a94d5c-baab-4af0-b012-e7a38b237370-istio-token\") pod \"92a94d5c-baab-4af0-b012-e7a38b237370\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " Jul 07 07:04:28.635527 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.635473 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "92a94d5c-baab-4af0-b012-e7a38b237370" (UID: "92a94d5c-baab-4af0-b012-e7a38b237370"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:04:28.635527 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.635498 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-credential-socket\") pod \"92a94d5c-baab-4af0-b012-e7a38b237370\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " Jul 07 07:04:28.635692 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.635555 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a94d5c-baab-4af0-b012-e7a38b237370-istiod-ca-cert\") pod \"92a94d5c-baab-4af0-b012-e7a38b237370\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " Jul 07 07:04:28.635692 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.635590 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-workload-certs\") pod \"92a94d5c-baab-4af0-b012-e7a38b237370\" (UID: \"92a94d5c-baab-4af0-b012-e7a38b237370\") " Jul 07 07:04:28.635800 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.635754 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "92a94d5c-baab-4af0-b012-e7a38b237370" (UID: "92a94d5c-baab-4af0-b012-e7a38b237370"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:04:28.635951 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.635932 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "92a94d5c-baab-4af0-b012-e7a38b237370" (UID: "92a94d5c-baab-4af0-b012-e7a38b237370"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:04:28.636025 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.635981 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-credential-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:28.636025 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.636001 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-workload-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:28.636025 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.636002 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92a94d5c-baab-4af0-b012-e7a38b237370-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "92a94d5c-baab-4af0-b012-e7a38b237370" (UID: "92a94d5c-baab-4af0-b012-e7a38b237370"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:04:28.636233 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.636206 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-istio-data" (OuterVolumeSpecName: "istio-data") pod "92a94d5c-baab-4af0-b012-e7a38b237370" (UID: "92a94d5c-baab-4af0-b012-e7a38b237370"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:04:28.637506 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.637483 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92a94d5c-baab-4af0-b012-e7a38b237370-kube-api-access-pzqch" (OuterVolumeSpecName: "kube-api-access-pzqch") pod "92a94d5c-baab-4af0-b012-e7a38b237370" (UID: "92a94d5c-baab-4af0-b012-e7a38b237370"). InnerVolumeSpecName "kube-api-access-pzqch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:04:28.637640 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.637627 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "92a94d5c-baab-4af0-b012-e7a38b237370" (UID: "92a94d5c-baab-4af0-b012-e7a38b237370"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:04:28.638144 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.638122 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92a94d5c-baab-4af0-b012-e7a38b237370-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "92a94d5c-baab-4af0-b012-e7a38b237370" (UID: "92a94d5c-baab-4af0-b012-e7a38b237370"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 07:04:28.638238 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.638136 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92a94d5c-baab-4af0-b012-e7a38b237370-istio-token" (OuterVolumeSpecName: "istio-token") pod "92a94d5c-baab-4af0-b012-e7a38b237370" (UID: "92a94d5c-baab-4af0-b012-e7a38b237370"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:04:28.638442 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.638424 2579 scope.go:117] "RemoveContainer" containerID="fd3460894dd97c70a4ff6b852804c9bee7bc84699ad91b8075e391e5926f1aaa" Jul 07 07:04:28.654520 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.654502 2579 scope.go:117] "RemoveContainer" containerID="611c889b321874a73c5cffd0eece245d10fdd4994798f4f22d9c9905d47c6279" Jul 07 07:04:28.654785 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:04:28.654763 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"611c889b321874a73c5cffd0eece245d10fdd4994798f4f22d9c9905d47c6279\": container with ID starting with 611c889b321874a73c5cffd0eece245d10fdd4994798f4f22d9c9905d47c6279 not found: ID does not exist" containerID="611c889b321874a73c5cffd0eece245d10fdd4994798f4f22d9c9905d47c6279" Jul 07 07:04:28.654838 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.654793 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"611c889b321874a73c5cffd0eece245d10fdd4994798f4f22d9c9905d47c6279"} err="failed to get container status \"611c889b321874a73c5cffd0eece245d10fdd4994798f4f22d9c9905d47c6279\": rpc error: code = NotFound desc = could not find container \"611c889b321874a73c5cffd0eece245d10fdd4994798f4f22d9c9905d47c6279\": container with ID starting with 611c889b321874a73c5cffd0eece245d10fdd4994798f4f22d9c9905d47c6279 not found: ID does not exist" Jul 07 07:04:28.654838 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.654811 2579 scope.go:117] "RemoveContainer" containerID="fd3460894dd97c70a4ff6b852804c9bee7bc84699ad91b8075e391e5926f1aaa" Jul 07 07:04:28.655037 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:04:28.655022 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd3460894dd97c70a4ff6b852804c9bee7bc84699ad91b8075e391e5926f1aaa\": container with ID starting with fd3460894dd97c70a4ff6b852804c9bee7bc84699ad91b8075e391e5926f1aaa not found: ID does not exist" containerID="fd3460894dd97c70a4ff6b852804c9bee7bc84699ad91b8075e391e5926f1aaa" Jul 07 07:04:28.655082 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.655041 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd3460894dd97c70a4ff6b852804c9bee7bc84699ad91b8075e391e5926f1aaa"} err="failed to get container status \"fd3460894dd97c70a4ff6b852804c9bee7bc84699ad91b8075e391e5926f1aaa\": rpc error: code = NotFound desc = could not find container \"fd3460894dd97c70a4ff6b852804c9bee7bc84699ad91b8075e391e5926f1aaa\": container with ID starting with fd3460894dd97c70a4ff6b852804c9bee7bc84699ad91b8075e391e5926f1aaa not found: ID does not exist" Jul 07 07:04:28.737426 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.737388 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a94d5c-baab-4af0-b012-e7a38b237370-istiod-ca-cert\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:28.737426 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.737418 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-workload-certs\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:28.737426 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.737426 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a94d5c-baab-4af0-b012-e7a38b237370-istio-podinfo\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:28.737426 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.737434 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-istio-envoy\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:28.737683 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.737443 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzqch\" (UniqueName: \"kubernetes.io/projected/92a94d5c-baab-4af0-b012-e7a38b237370-kube-api-access-pzqch\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:28.737683 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.737451 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a94d5c-baab-4af0-b012-e7a38b237370-istio-data\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:28.737683 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.737459 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a94d5c-baab-4af0-b012-e7a38b237370-istio-token\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:28.947393 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.947304 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n"] Jul 07 07:04:28.953522 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:28.953498 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-9b5a3c-openshift-default-86dd64b795-nld9n"] Jul 07 07:04:29.933532 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:29.933496 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:30.016315 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:30.016281 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:30.170785 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:30.170751 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r"] Jul 07 07:04:30.170978 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:30.170954 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" podUID="755b78f5-0288-4249-bcfe-bb5283c43cb6" containerName="istio-proxy" containerID="cri-o://8c1b35809b1f65ac23921cfe80dc90ea2a70da5036e0385ae3428e86519bf49c" gracePeriod=30 Jul 07 07:04:30.182033 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:30.182003 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:30.408326 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:30.408284 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:30.742299 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:30.742205 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:30.782676 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:30.782641 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92a94d5c-baab-4af0-b012-e7a38b237370" path="/var/lib/kubelet/pods/92a94d5c-baab-4af0-b012-e7a38b237370/volumes" Jul 07 07:04:31.057887 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.057852 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh"] Jul 07 07:04:31.058248 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.058234 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92a94d5c-baab-4af0-b012-e7a38b237370" containerName="istio-proxy" Jul 07 07:04:31.058248 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.058245 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="92a94d5c-baab-4af0-b012-e7a38b237370" containerName="istio-proxy" Jul 07 07:04:31.058332 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.058259 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="322fc4ff-e052-4e6f-b513-274dddec2e70" containerName="maas-api" Jul 07 07:04:31.058332 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.058264 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="322fc4ff-e052-4e6f-b513-274dddec2e70" containerName="maas-api" Jul 07 07:04:31.058332 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.058277 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92a94d5c-baab-4af0-b012-e7a38b237370" containerName="istio-proxy" Jul 07 07:04:31.058332 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.058286 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="92a94d5c-baab-4af0-b012-e7a38b237370" containerName="istio-proxy" Jul 07 07:04:31.058332 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.058298 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6fdedf8-15fd-4d5c-b010-f63d1a8387a3" containerName="maas-api" Jul 07 07:04:31.058332 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.058306 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6fdedf8-15fd-4d5c-b010-f63d1a8387a3" containerName="maas-api" Jul 07 07:04:31.058572 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.058412 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6fdedf8-15fd-4d5c-b010-f63d1a8387a3" containerName="maas-api" Jul 07 07:04:31.058572 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.058424 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="92a94d5c-baab-4af0-b012-e7a38b237370" containerName="istio-proxy" Jul 07 07:04:31.058572 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.058436 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="322fc4ff-e052-4e6f-b513-274dddec2e70" containerName="maas-api" Jul 07 07:04:31.058572 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.058445 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="92a94d5c-baab-4af0-b012-e7a38b237370" containerName="istio-proxy" Jul 07 07:04:31.063252 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.063231 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.066174 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.066154 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-98f550-openshift-default-dockercfg-bnh49\"" Jul 07 07:04:31.076571 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.076552 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh"] Jul 07 07:04:31.104908 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.104762 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:31.159500 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.159459 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa946518-5816-42af-9923-70d6e54de4e2-istio-token\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.159804 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.159766 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-workload-certs\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.159933 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.159844 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2qsh\" (UniqueName: \"kubernetes.io/projected/aa946518-5816-42af-9923-70d6e54de4e2-kube-api-access-v2qsh\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.159933 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.159877 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-workload-socket\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.159933 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.159899 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa946518-5816-42af-9923-70d6e54de4e2-istiod-ca-cert\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.159933 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.159930 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-credential-socket\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.160215 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.159960 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-istio-envoy\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.160215 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.159987 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-istio-data\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.160215 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.160028 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa946518-5816-42af-9923-70d6e54de4e2-istio-podinfo\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.261443 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.261410 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-istio-envoy\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.261443 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.261446 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-istio-data\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.261694 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.261472 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa946518-5816-42af-9923-70d6e54de4e2-istio-podinfo\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.261694 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.261529 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa946518-5816-42af-9923-70d6e54de4e2-istio-token\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.261694 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.261563 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-workload-certs\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.261694 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.261611 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2qsh\" (UniqueName: \"kubernetes.io/projected/aa946518-5816-42af-9923-70d6e54de4e2-kube-api-access-v2qsh\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.261694 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.261637 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-workload-socket\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.261694 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.261659 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa946518-5816-42af-9923-70d6e54de4e2-istiod-ca-cert\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.261694 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.261694 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-credential-socket\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.262048 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.261897 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-istio-data\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.262048 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.261967 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-workload-certs\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.262126 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.262087 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-credential-socket\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.262217 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.262176 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-workload-socket\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.262485 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.262465 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa946518-5816-42af-9923-70d6e54de4e2-istiod-ca-cert\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.263640 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.263616 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-istio-envoy\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.263896 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.263878 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa946518-5816-42af-9923-70d6e54de4e2-istio-podinfo\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.274745 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.274722 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa946518-5816-42af-9923-70d6e54de4e2-istio-token\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.275952 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.275930 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2qsh\" (UniqueName: \"kubernetes.io/projected/aa946518-5816-42af-9923-70d6e54de4e2-kube-api-access-v2qsh\") pod \"e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.373829 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.373771 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:31.502591 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.502553 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:31.512548 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.512527 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh"] Jul 07 07:04:31.513869 ip-10-0-130-176 kubenswrapper[2579]: W0707 07:04:31.513845 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa946518_5816_42af_9923_70d6e54de4e2.slice/crio-cce2d6947e5b2e6ccb5e28a37c11fa733b6d0ca43e3fe7e54ab018f2b762eb8f WatchSource:0}: Error finding container cce2d6947e5b2e6ccb5e28a37c11fa733b6d0ca43e3fe7e54ab018f2b762eb8f: Status 404 returned error can't find the container with id cce2d6947e5b2e6ccb5e28a37c11fa733b6d0ca43e3fe7e54ab018f2b762eb8f Jul 07 07:04:31.515919 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.515888 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:04:31.516020 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.515949 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:04:31.516020 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.515976 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:04:31.648031 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.647935 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" event={"ID":"aa946518-5816-42af-9923-70d6e54de4e2","Type":"ContainerStarted","Data":"a272d98791dc63c9e9694dbf3c8a087a8ef136bb25ef90adb7b3fbc33f7a0c3a"} Jul 07 07:04:31.648031 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.647998 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" event={"ID":"aa946518-5816-42af-9923-70d6e54de4e2","Type":"ContainerStarted","Data":"cce2d6947e5b2e6ccb5e28a37c11fa733b6d0ca43e3fe7e54ab018f2b762eb8f"} Jul 07 07:04:31.669431 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:31.669384 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" podStartSLOduration=0.669355061 podStartE2EDuration="669.355061ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:04:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:04:31.667765721 +0000 UTC m=+1947.507606691" watchObservedRunningTime="2026-07-07 07:04:31.669355061 +0000 UTC m=+1947.509196094" Jul 07 07:04:32.374472 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:32.374434 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:32.379249 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:32.379228 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:32.652872 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:32.652779 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:32.653705 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:32.653682 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:33.318338 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.318300 2579 patch_prober.go:28] interesting pod/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 07:04:33.318542 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.318357 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" podUID="755b78f5-0288-4249-bcfe-bb5283c43cb6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 07:04:33.610043 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.609932 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:33.690082 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.690050 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm"] Jul 07 07:04:33.693576 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.693558 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:33.696978 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.696950 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-98f550-serving-cert\"" Jul 07 07:04:33.697095 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.696964 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-754v8\"" Jul 07 07:04:33.717849 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.717821 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm"] Jul 07 07:04:33.722746 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.722715 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:33.788587 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.788545 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgmbf\" (UniqueName: \"kubernetes.io/projected/e4ee35a1-562c-4867-8560-0ab0b88b5490-kube-api-access-cgmbf\") pod \"maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm\" (UID: \"e4ee35a1-562c-4867-8560-0ab0b88b5490\") " pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:33.788780 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.788643 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4ee35a1-562c-4867-8560-0ab0b88b5490-openshift-service-ca\") pod \"maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm\" (UID: \"e4ee35a1-562c-4867-8560-0ab0b88b5490\") " pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:33.788780 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.788721 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4ee35a1-562c-4867-8560-0ab0b88b5490-maas-api-tls\") pod \"maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm\" (UID: \"e4ee35a1-562c-4867-8560-0ab0b88b5490\") " pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:33.824286 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.824248 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:33.889776 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.889699 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgmbf\" (UniqueName: \"kubernetes.io/projected/e4ee35a1-562c-4867-8560-0ab0b88b5490-kube-api-access-cgmbf\") pod \"maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm\" (UID: \"e4ee35a1-562c-4867-8560-0ab0b88b5490\") " pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:33.889776 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.889747 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4ee35a1-562c-4867-8560-0ab0b88b5490-openshift-service-ca\") pod \"maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm\" (UID: \"e4ee35a1-562c-4867-8560-0ab0b88b5490\") " pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:33.889964 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.889779 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4ee35a1-562c-4867-8560-0ab0b88b5490-maas-api-tls\") pod \"maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm\" (UID: \"e4ee35a1-562c-4867-8560-0ab0b88b5490\") " pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:33.889964 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:04:33.889870 2579 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-98f550-serving-cert: secret "maas-api-e2e-api-a-98f550-serving-cert" not found Jul 07 07:04:33.889964 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:04:33.889920 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4ee35a1-562c-4867-8560-0ab0b88b5490-maas-api-tls podName:e4ee35a1-562c-4867-8560-0ab0b88b5490 nodeName:}" failed. No retries permitted until 2026-07-07 07:04:34.389906602 +0000 UTC m=+1950.229747548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e4ee35a1-562c-4867-8560-0ab0b88b5490-maas-api-tls") pod "maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" (UID: "e4ee35a1-562c-4867-8560-0ab0b88b5490") : secret "maas-api-e2e-api-a-98f550-serving-cert" not found Jul 07 07:04:33.890389 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.890347 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4ee35a1-562c-4867-8560-0ab0b88b5490-openshift-service-ca\") pod \"maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm\" (UID: \"e4ee35a1-562c-4867-8560-0ab0b88b5490\") " pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:33.899758 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:33.899734 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgmbf\" (UniqueName: \"kubernetes.io/projected/e4ee35a1-562c-4867-8560-0ab0b88b5490-kube-api-access-cgmbf\") pod \"maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm\" (UID: \"e4ee35a1-562c-4867-8560-0ab0b88b5490\") " pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:34.296234 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:34.296196 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:34.394879 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:34.394839 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4ee35a1-562c-4867-8560-0ab0b88b5490-maas-api-tls\") pod \"maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm\" (UID: \"e4ee35a1-562c-4867-8560-0ab0b88b5490\") " pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:34.397480 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:34.397455 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:34.397632 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:34.397586 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4ee35a1-562c-4867-8560-0ab0b88b5490-maas-api-tls\") pod \"maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm\" (UID: \"e4ee35a1-562c-4867-8560-0ab0b88b5490\") " pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:34.604584 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:34.604495 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:34.747080 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:34.747057 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm"] Jul 07 07:04:34.752425 ip-10-0-130-176 kubenswrapper[2579]: W0707 07:04:34.752391 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ee35a1_562c_4867_8560_0ab0b88b5490.slice/crio-c56f479668fa62173fa9b5877b70beffbd79cc20e08a570509c01fb82c19c36c WatchSource:0}: Error finding container c56f479668fa62173fa9b5877b70beffbd79cc20e08a570509c01fb82c19c36c: Status 404 returned error can't find the container with id c56f479668fa62173fa9b5877b70beffbd79cc20e08a570509c01fb82c19c36c Jul 07 07:04:35.425709 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.425687 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:04:35.506332 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.506247 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-podinfo\") pod \"755b78f5-0288-4249-bcfe-bb5283c43cb6\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " Jul 07 07:04:35.506486 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.506407 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-workload-certs\") pod \"755b78f5-0288-4249-bcfe-bb5283c43cb6\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " Jul 07 07:04:35.506486 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.506445 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-workload-socket\") pod \"755b78f5-0288-4249-bcfe-bb5283c43cb6\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " Jul 07 07:04:35.506486 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.506478 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m87xq\" (UniqueName: \"kubernetes.io/projected/755b78f5-0288-4249-bcfe-bb5283c43cb6-kube-api-access-m87xq\") pod \"755b78f5-0288-4249-bcfe-bb5283c43cb6\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " Jul 07 07:04:35.506661 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.506545 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-envoy\") pod \"755b78f5-0288-4249-bcfe-bb5283c43cb6\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " Jul 07 07:04:35.506661 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.506568 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-data\") pod \"755b78f5-0288-4249-bcfe-bb5283c43cb6\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " Jul 07 07:04:35.506661 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.506589 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-token\") pod \"755b78f5-0288-4249-bcfe-bb5283c43cb6\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " Jul 07 07:04:35.506661 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.506616 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/755b78f5-0288-4249-bcfe-bb5283c43cb6-istiod-ca-cert\") pod \"755b78f5-0288-4249-bcfe-bb5283c43cb6\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " Jul 07 07:04:35.506661 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.506651 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-credential-socket\") pod \"755b78f5-0288-4249-bcfe-bb5283c43cb6\" (UID: \"755b78f5-0288-4249-bcfe-bb5283c43cb6\") " Jul 07 07:04:35.506914 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.506736 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "755b78f5-0288-4249-bcfe-bb5283c43cb6" (UID: "755b78f5-0288-4249-bcfe-bb5283c43cb6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:04:35.506914 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.506750 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "755b78f5-0288-4249-bcfe-bb5283c43cb6" (UID: "755b78f5-0288-4249-bcfe-bb5283c43cb6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:04:35.507026 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.506968 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "755b78f5-0288-4249-bcfe-bb5283c43cb6" (UID: "755b78f5-0288-4249-bcfe-bb5283c43cb6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:04:35.507102 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.507053 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-workload-certs\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:35.507102 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.507070 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-workload-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:35.507102 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.507084 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-credential-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:35.507313 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.507284 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/755b78f5-0288-4249-bcfe-bb5283c43cb6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "755b78f5-0288-4249-bcfe-bb5283c43cb6" (UID: "755b78f5-0288-4249-bcfe-bb5283c43cb6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:04:35.507494 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.507463 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-data" (OuterVolumeSpecName: "istio-data") pod "755b78f5-0288-4249-bcfe-bb5283c43cb6" (UID: "755b78f5-0288-4249-bcfe-bb5283c43cb6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:04:35.508701 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.508676 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "755b78f5-0288-4249-bcfe-bb5283c43cb6" (UID: "755b78f5-0288-4249-bcfe-bb5283c43cb6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 07:04:35.508794 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.508778 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "755b78f5-0288-4249-bcfe-bb5283c43cb6" (UID: "755b78f5-0288-4249-bcfe-bb5283c43cb6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:04:35.508835 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.508811 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-token" (OuterVolumeSpecName: "istio-token") pod "755b78f5-0288-4249-bcfe-bb5283c43cb6" (UID: "755b78f5-0288-4249-bcfe-bb5283c43cb6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:04:35.508868 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.508815 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/755b78f5-0288-4249-bcfe-bb5283c43cb6-kube-api-access-m87xq" (OuterVolumeSpecName: "kube-api-access-m87xq") pod "755b78f5-0288-4249-bcfe-bb5283c43cb6" (UID: "755b78f5-0288-4249-bcfe-bb5283c43cb6"). InnerVolumeSpecName "kube-api-access-m87xq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:04:35.608231 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.608195 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/755b78f5-0288-4249-bcfe-bb5283c43cb6-istiod-ca-cert\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:35.608231 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.608224 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-podinfo\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:35.608231 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.608233 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m87xq\" (UniqueName: \"kubernetes.io/projected/755b78f5-0288-4249-bcfe-bb5283c43cb6-kube-api-access-m87xq\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:35.608478 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.608242 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-envoy\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:35.608478 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.608250 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-data\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:35.608478 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.608257 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/755b78f5-0288-4249-bcfe-bb5283c43cb6-istio-token\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:35.671894 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.671859 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" event={"ID":"e4ee35a1-562c-4867-8560-0ab0b88b5490","Type":"ContainerStarted","Data":"28e3f06e97b7e3d5315e33ed22e2a2f64adffd9fdf83bcb04563fd05ff52bff9"} Jul 07 07:04:35.672076 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.671960 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:35.672076 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.671980 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" event={"ID":"e4ee35a1-562c-4867-8560-0ab0b88b5490","Type":"ContainerStarted","Data":"c56f479668fa62173fa9b5877b70beffbd79cc20e08a570509c01fb82c19c36c"} Jul 07 07:04:35.673427 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.673402 2579 generic.go:356] "Generic (PLEG): container finished" podID="755b78f5-0288-4249-bcfe-bb5283c43cb6" containerID="8c1b35809b1f65ac23921cfe80dc90ea2a70da5036e0385ae3428e86519bf49c" exitCode=0 Jul 07 07:04:35.673535 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.673482 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" Jul 07 07:04:35.673639 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.673476 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" event={"ID":"755b78f5-0288-4249-bcfe-bb5283c43cb6","Type":"ContainerDied","Data":"8c1b35809b1f65ac23921cfe80dc90ea2a70da5036e0385ae3428e86519bf49c"} Jul 07 07:04:35.673639 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.673593 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r" event={"ID":"755b78f5-0288-4249-bcfe-bb5283c43cb6","Type":"ContainerDied","Data":"5aa8f1a4b323dfc1aea3b9bdff7c8d7f8c013b8c85c60b86c2878ad6c2b9cf25"} Jul 07 07:04:35.673639 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.673616 2579 scope.go:117] "RemoveContainer" containerID="8c1b35809b1f65ac23921cfe80dc90ea2a70da5036e0385ae3428e86519bf49c" Jul 07 07:04:35.682522 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.682504 2579 scope.go:117] "RemoveContainer" containerID="d6fc4f33b08e1eb5746d198ee39dbfa3dbbf160105ff6894195382fcc64782c5" Jul 07 07:04:35.690687 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.690634 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" podStartSLOduration=2.256201936 podStartE2EDuration="2.690617983s" totalImagesPullingTime="434.416047ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:04:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:04:35.68861412 +0000 UTC m=+1951.528455101" watchObservedRunningTime="2026-07-07 07:04:35.690617983 +0000 UTC m=+1951.530458953" Jul 07 07:04:35.691912 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.691892 2579 scope.go:117] "RemoveContainer" containerID="8c1b35809b1f65ac23921cfe80dc90ea2a70da5036e0385ae3428e86519bf49c" Jul 07 07:04:35.692167 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:04:35.692149 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c1b35809b1f65ac23921cfe80dc90ea2a70da5036e0385ae3428e86519bf49c\": container with ID starting with 8c1b35809b1f65ac23921cfe80dc90ea2a70da5036e0385ae3428e86519bf49c not found: ID does not exist" containerID="8c1b35809b1f65ac23921cfe80dc90ea2a70da5036e0385ae3428e86519bf49c" Jul 07 07:04:35.692244 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.692173 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c1b35809b1f65ac23921cfe80dc90ea2a70da5036e0385ae3428e86519bf49c"} err="failed to get container status \"8c1b35809b1f65ac23921cfe80dc90ea2a70da5036e0385ae3428e86519bf49c\": rpc error: code = NotFound desc = could not find container \"8c1b35809b1f65ac23921cfe80dc90ea2a70da5036e0385ae3428e86519bf49c\": container with ID starting with 8c1b35809b1f65ac23921cfe80dc90ea2a70da5036e0385ae3428e86519bf49c not found: ID does not exist" Jul 07 07:04:35.692244 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.692189 2579 scope.go:117] "RemoveContainer" containerID="d6fc4f33b08e1eb5746d198ee39dbfa3dbbf160105ff6894195382fcc64782c5" Jul 07 07:04:35.692427 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:04:35.692408 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6fc4f33b08e1eb5746d198ee39dbfa3dbbf160105ff6894195382fcc64782c5\": container with ID starting with d6fc4f33b08e1eb5746d198ee39dbfa3dbbf160105ff6894195382fcc64782c5 not found: ID does not exist" containerID="d6fc4f33b08e1eb5746d198ee39dbfa3dbbf160105ff6894195382fcc64782c5" Jul 07 07:04:35.692486 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.692432 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6fc4f33b08e1eb5746d198ee39dbfa3dbbf160105ff6894195382fcc64782c5"} err="failed to get container status \"d6fc4f33b08e1eb5746d198ee39dbfa3dbbf160105ff6894195382fcc64782c5\": rpc error: code = NotFound desc = could not find container \"d6fc4f33b08e1eb5746d198ee39dbfa3dbbf160105ff6894195382fcc64782c5\": container with ID starting with d6fc4f33b08e1eb5746d198ee39dbfa3dbbf160105ff6894195382fcc64782c5 not found: ID does not exist" Jul 07 07:04:35.704236 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.704213 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r"] Jul 07 07:04:35.708123 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:35.708102 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-da1ed1-openshift-default-fb546f579-xgk4r"] Jul 07 07:04:36.782806 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:36.782776 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="755b78f5-0288-4249-bcfe-bb5283c43cb6" path="/var/lib/kubelet/pods/755b78f5-0288-4249-bcfe-bb5283c43cb6/volumes" Jul 07 07:04:38.088887 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.088848 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:38.530707 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.530673 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:38.784441 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.784345 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k"] Jul 07 07:04:38.784960 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.784929 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="755b78f5-0288-4249-bcfe-bb5283c43cb6" containerName="istio-proxy" Jul 07 07:04:38.784960 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.784951 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="755b78f5-0288-4249-bcfe-bb5283c43cb6" containerName="istio-proxy" Jul 07 07:04:38.785136 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.785056 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="755b78f5-0288-4249-bcfe-bb5283c43cb6" containerName="istio-proxy" Jul 07 07:04:38.785136 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.785074 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="755b78f5-0288-4249-bcfe-bb5283c43cb6" containerName="istio-proxy" Jul 07 07:04:38.788095 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.788072 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:38.790880 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.790861 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-50d762-openshift-default-dockercfg-jlkxk\"" Jul 07 07:04:38.799785 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.799765 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k"] Jul 07 07:04:38.810966 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.810940 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:38.942450 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.942413 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-podinfo\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:38.942631 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.942469 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-envoy\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:38.942631 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.942509 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-credential-socket\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:38.942631 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.942535 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-workload-certs\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:38.942631 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.942572 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istiod-ca-cert\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:38.942631 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.942589 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-token\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:38.942888 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.942664 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mplg4\" (UniqueName: \"kubernetes.io/projected/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-kube-api-access-mplg4\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:38.942888 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.942719 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-data\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:38.942888 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:38.942764 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-workload-socket\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.043726 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.043639 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istiod-ca-cert\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.043726 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.043673 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-token\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.043726 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.043705 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mplg4\" (UniqueName: \"kubernetes.io/projected/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-kube-api-access-mplg4\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.044038 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.043737 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-data\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.044038 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.043764 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-workload-socket\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.044038 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.043802 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-podinfo\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.044038 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.043846 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-envoy\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.044038 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.043882 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-credential-socket\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.044038 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.043915 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-workload-certs\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.044248 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.044223 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-data\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.044333 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.044312 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-workload-certs\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.044415 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.044327 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-credential-socket\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.044485 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.044427 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istiod-ca-cert\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.044485 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.044464 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-workload-socket\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.046238 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.046214 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-envoy\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.046388 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.046354 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-podinfo\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.052092 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.052065 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-token\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.052256 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.052239 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mplg4\" (UniqueName: \"kubernetes.io/projected/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-kube-api-access-mplg4\") pod \"e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.099841 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.099816 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:39.242511 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.242480 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k"] Jul 07 07:04:39.243616 ip-10-0-130-176 kubenswrapper[2579]: W0707 07:04:39.243583 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ead2c4_dd28_4349_a5b9_4fa6cad7dcab.slice/crio-b6e02d3f4174780e5e3ca35f2eeae88cf506296f583d06755eec233b1737106c WatchSource:0}: Error finding container b6e02d3f4174780e5e3ca35f2eeae88cf506296f583d06755eec233b1737106c: Status 404 returned error can't find the container with id b6e02d3f4174780e5e3ca35f2eeae88cf506296f583d06755eec233b1737106c Jul 07 07:04:39.246127 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.246090 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:04:39.246236 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.246172 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:04:39.246236 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.246218 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:04:39.324283 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.324254 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:39.696189 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.696104 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" event={"ID":"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab","Type":"ContainerStarted","Data":"b69f2f1e1fbf0c2488182ddf5870f9db959482ec1bbf520b880f374459159876"} Jul 07 07:04:39.696189 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.696160 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" event={"ID":"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab","Type":"ContainerStarted","Data":"b6e02d3f4174780e5e3ca35f2eeae88cf506296f583d06755eec233b1737106c"} Jul 07 07:04:39.720102 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:39.720049 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" podStartSLOduration=1.7200334769999999 podStartE2EDuration="1.720033477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:04:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:04:39.71848678 +0000 UTC m=+1955.558327748" watchObservedRunningTime="2026-07-07 07:04:39.720033477 +0000 UTC m=+1955.559874498" Jul 07 07:04:40.099985 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:40.099945 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:40.104872 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:40.104843 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:40.701840 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:40.701808 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:40.702706 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:40.702689 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:41.634630 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.634580 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:41.685190 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.685163 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:41.707117 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.707082 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8"] Jul 07 07:04:41.707540 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.707523 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="755b78f5-0288-4249-bcfe-bb5283c43cb6" containerName="istio-proxy" Jul 07 07:04:41.707642 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.707542 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="755b78f5-0288-4249-bcfe-bb5283c43cb6" containerName="istio-proxy" Jul 07 07:04:41.710704 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.710684 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" Jul 07 07:04:41.713533 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.713510 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-50d762-serving-cert\"" Jul 07 07:04:41.715905 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.715878 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:41.725353 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.725329 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8"] Jul 07 07:04:41.784677 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.784626 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:41.873888 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.873848 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kshxc\" (UniqueName: \"kubernetes.io/projected/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-kube-api-access-kshxc\") pod \"maas-api-e2e-api-b-50d762-558f45446c-r6bs8\" (UID: \"8b4c8b70-90eb-42df-bbd7-7f9566f2613a\") " pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" Jul 07 07:04:41.874099 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.873903 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-openshift-service-ca\") pod \"maas-api-e2e-api-b-50d762-558f45446c-r6bs8\" (UID: \"8b4c8b70-90eb-42df-bbd7-7f9566f2613a\") " pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" Jul 07 07:04:41.874099 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.873967 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-maas-api-tls\") pod \"maas-api-e2e-api-b-50d762-558f45446c-r6bs8\" (UID: \"8b4c8b70-90eb-42df-bbd7-7f9566f2613a\") " pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" Jul 07 07:04:41.883762 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.883730 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:41.975236 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.975147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-openshift-service-ca\") pod \"maas-api-e2e-api-b-50d762-558f45446c-r6bs8\" (UID: \"8b4c8b70-90eb-42df-bbd7-7f9566f2613a\") " pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" Jul 07 07:04:41.975236 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.975200 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-maas-api-tls\") pod \"maas-api-e2e-api-b-50d762-558f45446c-r6bs8\" (UID: \"8b4c8b70-90eb-42df-bbd7-7f9566f2613a\") " pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" Jul 07 07:04:41.975448 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.975271 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kshxc\" (UniqueName: \"kubernetes.io/projected/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-kube-api-access-kshxc\") pod \"maas-api-e2e-api-b-50d762-558f45446c-r6bs8\" (UID: \"8b4c8b70-90eb-42df-bbd7-7f9566f2613a\") " pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" Jul 07 07:04:41.975865 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.975842 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-openshift-service-ca\") pod \"maas-api-e2e-api-b-50d762-558f45446c-r6bs8\" (UID: \"8b4c8b70-90eb-42df-bbd7-7f9566f2613a\") " pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" Jul 07 07:04:41.977653 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.977634 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-maas-api-tls\") pod \"maas-api-e2e-api-b-50d762-558f45446c-r6bs8\" (UID: \"8b4c8b70-90eb-42df-bbd7-7f9566f2613a\") " pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" Jul 07 07:04:41.984452 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:41.984432 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kshxc\" (UniqueName: \"kubernetes.io/projected/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-kube-api-access-kshxc\") pod \"maas-api-e2e-api-b-50d762-558f45446c-r6bs8\" (UID: \"8b4c8b70-90eb-42df-bbd7-7f9566f2613a\") " pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" Jul 07 07:04:42.021990 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:42.021964 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" Jul 07 07:04:42.174198 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:42.174162 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:42.203149 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:42.203122 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8"] Jul 07 07:04:42.204596 ip-10-0-130-176 kubenswrapper[2579]: W0707 07:04:42.204548 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4c8b70_90eb_42df_bbd7_7f9566f2613a.slice/crio-e267a57d40b91a9eb7720e57e65cc7aad823357e641be5fa8afe4b7041f589fe WatchSource:0}: Error finding container e267a57d40b91a9eb7720e57e65cc7aad823357e641be5fa8afe4b7041f589fe: Status 404 returned error can't find the container with id e267a57d40b91a9eb7720e57e65cc7aad823357e641be5fa8afe4b7041f589fe Jul 07 07:04:42.271926 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:42.271793 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:42.715259 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:42.715229 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" event={"ID":"8b4c8b70-90eb-42df-bbd7-7f9566f2613a","Type":"ContainerStarted","Data":"e267a57d40b91a9eb7720e57e65cc7aad823357e641be5fa8afe4b7041f589fe"} Jul 07 07:04:43.723152 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:43.723116 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" event={"ID":"8b4c8b70-90eb-42df-bbd7-7f9566f2613a","Type":"ContainerStarted","Data":"13bfed68fbd6164ef97fa8ad964099f2add1a6a03eca3c20d51327e7a89db11e"} Jul 07 07:04:43.723560 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:43.723189 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" Jul 07 07:04:43.747518 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:43.747399 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" podStartSLOduration=2.24449538 podStartE2EDuration="2.747382909s" totalImagesPullingTime="502.887529ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:04:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:04:43.746691665 +0000 UTC m=+1959.586532631" watchObservedRunningTime="2026-07-07 07:04:43.747382909 +0000 UTC m=+1959.587223870" Jul 07 07:04:45.653128 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:45.653087 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:45.755040 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:45.754996 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:46.122184 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:46.122108 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:46.742113 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:46.742083 2579 generic.go:356] "Generic (PLEG): container finished" podID="aa946518-5816-42af-9923-70d6e54de4e2" containerID="a272d98791dc63c9e9694dbf3c8a087a8ef136bb25ef90adb7b3fbc33f7a0c3a" exitCode=0 Jul 07 07:04:46.742518 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:46.742157 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" event={"ID":"aa946518-5816-42af-9923-70d6e54de4e2","Type":"ContainerDied","Data":"a272d98791dc63c9e9694dbf3c8a087a8ef136bb25ef90adb7b3fbc33f7a0c3a"} Jul 07 07:04:46.742565 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:46.742547 2579 scope.go:117] "RemoveContainer" containerID="a272d98791dc63c9e9694dbf3c8a087a8ef136bb25ef90adb7b3fbc33f7a0c3a" Jul 07 07:04:46.743482 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:46.743455 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:04:46.743541 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:46.743523 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:04:46.743581 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:46.743555 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:04:47.653042 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:47.653008 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:47.754727 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:47.754694 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" event={"ID":"aa946518-5816-42af-9923-70d6e54de4e2","Type":"ContainerStarted","Data":"793e8fa5486051a83e188d2f9521a58f23a25190337160cb97132fa4953a22b2"} Jul 07 07:04:48.374458 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:48.374419 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:48.375566 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:48.375525 2579 patch_prober.go:28] interesting pod/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 07 07:04:48.375690 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:48.375595 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" podUID="aa946518-5816-42af-9923-70d6e54de4e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 07 07:04:49.374430 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:49.374390 2579 patch_prober.go:28] interesting pod/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 07 07:04:49.375042 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:49.375010 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" podUID="aa946518-5816-42af-9923-70d6e54de4e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 07 07:04:49.745636 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:49.744618 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" Jul 07 07:04:50.374781 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:50.374736 2579 patch_prober.go:28] interesting pod/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 07 07:04:50.375285 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:50.374808 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" podUID="aa946518-5816-42af-9923-70d6e54de4e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 07 07:04:51.374315 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:51.374267 2579 patch_prober.go:28] interesting pod/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 07 07:04:51.374553 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:51.374337 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" podUID="aa946518-5816-42af-9923-70d6e54de4e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 07 07:04:52.378752 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:52.378721 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:52.379223 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:52.379071 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:52.379839 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:52.379816 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:04:54.086339 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:54.086296 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:54.446161 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:54.446075 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:54.547030 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:54.546997 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:54.659175 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:54.659128 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm"] Jul 07 07:04:54.659445 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:54.659414 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" podUID="e4ee35a1-562c-4867-8560-0ab0b88b5490" containerName="maas-api" containerID="cri-o://28e3f06e97b7e3d5315e33ed22e2a2f64adffd9fdf83bcb04563fd05ff52bff9" gracePeriod=30 Jul 07 07:04:54.680601 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:54.680571 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:04:54.811335 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:54.811306 2579 generic.go:356] "Generic (PLEG): container finished" podID="e4ee35a1-562c-4867-8560-0ab0b88b5490" containerID="28e3f06e97b7e3d5315e33ed22e2a2f64adffd9fdf83bcb04563fd05ff52bff9" exitCode=0 Jul 07 07:04:54.811513 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:54.811384 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" event={"ID":"e4ee35a1-562c-4867-8560-0ab0b88b5490","Type":"ContainerDied","Data":"28e3f06e97b7e3d5315e33ed22e2a2f64adffd9fdf83bcb04563fd05ff52bff9"} Jul 07 07:04:54.910749 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:54.910725 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:55.007202 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.007173 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4ee35a1-562c-4867-8560-0ab0b88b5490-maas-api-tls\") pod \"e4ee35a1-562c-4867-8560-0ab0b88b5490\" (UID: \"e4ee35a1-562c-4867-8560-0ab0b88b5490\") " Jul 07 07:04:55.007378 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.007226 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgmbf\" (UniqueName: \"kubernetes.io/projected/e4ee35a1-562c-4867-8560-0ab0b88b5490-kube-api-access-cgmbf\") pod \"e4ee35a1-562c-4867-8560-0ab0b88b5490\" (UID: \"e4ee35a1-562c-4867-8560-0ab0b88b5490\") " Jul 07 07:04:55.007378 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.007343 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4ee35a1-562c-4867-8560-0ab0b88b5490-openshift-service-ca\") pod \"e4ee35a1-562c-4867-8560-0ab0b88b5490\" (UID: \"e4ee35a1-562c-4867-8560-0ab0b88b5490\") " Jul 07 07:04:55.007660 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.007630 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4ee35a1-562c-4867-8560-0ab0b88b5490-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e4ee35a1-562c-4867-8560-0ab0b88b5490" (UID: "e4ee35a1-562c-4867-8560-0ab0b88b5490"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:04:55.007785 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.007731 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4ee35a1-562c-4867-8560-0ab0b88b5490-openshift-service-ca\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:55.009351 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.009328 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4ee35a1-562c-4867-8560-0ab0b88b5490-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e4ee35a1-562c-4867-8560-0ab0b88b5490" (UID: "e4ee35a1-562c-4867-8560-0ab0b88b5490"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:04:55.009443 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.009328 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4ee35a1-562c-4867-8560-0ab0b88b5490-kube-api-access-cgmbf" (OuterVolumeSpecName: "kube-api-access-cgmbf") pod "e4ee35a1-562c-4867-8560-0ab0b88b5490" (UID: "e4ee35a1-562c-4867-8560-0ab0b88b5490"). InnerVolumeSpecName "kube-api-access-cgmbf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:04:55.108430 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.108339 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4ee35a1-562c-4867-8560-0ab0b88b5490-maas-api-tls\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:55.108430 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.108396 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgmbf\" (UniqueName: \"kubernetes.io/projected/e4ee35a1-562c-4867-8560-0ab0b88b5490-kube-api-access-cgmbf\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:04:55.703424 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.703387 2579 patch_prober.go:28] interesting pod/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 07 07:04:55.703599 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.703450 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" podUID="f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 07 07:04:55.817663 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.817633 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" Jul 07 07:04:55.817869 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.817670 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm" event={"ID":"e4ee35a1-562c-4867-8560-0ab0b88b5490","Type":"ContainerDied","Data":"c56f479668fa62173fa9b5877b70beffbd79cc20e08a570509c01fb82c19c36c"} Jul 07 07:04:55.817869 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.817728 2579 scope.go:117] "RemoveContainer" containerID="28e3f06e97b7e3d5315e33ed22e2a2f64adffd9fdf83bcb04563fd05ff52bff9" Jul 07 07:04:55.819355 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.819323 2579 generic.go:356] "Generic (PLEG): container finished" podID="f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" containerID="b69f2f1e1fbf0c2488182ddf5870f9db959482ec1bbf520b880f374459159876" exitCode=0 Jul 07 07:04:55.819474 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.819417 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" event={"ID":"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab","Type":"ContainerDied","Data":"b69f2f1e1fbf0c2488182ddf5870f9db959482ec1bbf520b880f374459159876"} Jul 07 07:04:55.819904 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.819887 2579 scope.go:117] "RemoveContainer" containerID="b69f2f1e1fbf0c2488182ddf5870f9db959482ec1bbf520b880f374459159876" Jul 07 07:04:55.820994 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.820967 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:04:55.821079 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.821045 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:04:55.821133 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.821094 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:04:55.861508 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.861480 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm"] Jul 07 07:04:55.866715 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:55.866686 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-98f550-7c776bf6d5-vmrcm"] Jul 07 07:04:56.783302 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:56.783228 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4ee35a1-562c-4867-8560-0ab0b88b5490" path="/var/lib/kubelet/pods/e4ee35a1-562c-4867-8560-0ab0b88b5490/volumes" Jul 07 07:04:56.826731 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:56.826694 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" event={"ID":"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab","Type":"ContainerStarted","Data":"4d93053f97e01a986c463358376a7948a830e0b4df65615a752b1886683c78eb"} Jul 07 07:04:57.101007 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:57.100913 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:04:57.101179 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:57.101156 2579 patch_prober.go:28] interesting pod/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 07 07:04:57.101223 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:57.101205 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" podUID="f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 07 07:04:58.101125 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:58.101089 2579 patch_prober.go:28] interesting pod/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 07 07:04:58.101520 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:58.101158 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" podUID="f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 07 07:04:59.101027 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:59.100992 2579 patch_prober.go:28] interesting pod/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 07 07:04:59.101204 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:04:59.101060 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" podUID="f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 07 07:05:00.100576 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:00.100533 2579 patch_prober.go:28] interesting pod/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 07 07:05:00.100774 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:00.100600 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" podUID="f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 07 07:05:00.514877 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:00.514768 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:00.593469 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:00.593429 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:00.804881 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:00.804850 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh"] Jul 07 07:05:00.805061 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:00.805047 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" podUID="aa946518-5816-42af-9923-70d6e54de4e2" containerName="istio-proxy" containerID="cri-o://793e8fa5486051a83e188d2f9521a58f23a25190337160cb97132fa4953a22b2" gracePeriod=30 Jul 07 07:05:00.809165 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:00.809141 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:01.039889 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:01.039853 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:01.104652 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:01.104558 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:05:01.105019 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:01.104993 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:05:01.105723 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:01.105703 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:05:01.207702 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:01.207661 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:01.598977 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:01.598942 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:01.692789 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:01.692754 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:01.828716 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:01.828687 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8"] Jul 07 07:05:01.828969 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:01.828936 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" podUID="8b4c8b70-90eb-42df-bbd7-7f9566f2613a" containerName="maas-api" containerID="cri-o://13bfed68fbd6164ef97fa8ad964099f2add1a6a03eca3c20d51327e7a89db11e" gracePeriod=30 Jul 07 07:05:01.882552 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:01.882468 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:02.070853 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.070827 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" Jul 07 07:05:02.181441 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.181325 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kshxc\" (UniqueName: \"kubernetes.io/projected/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-kube-api-access-kshxc\") pod \"8b4c8b70-90eb-42df-bbd7-7f9566f2613a\" (UID: \"8b4c8b70-90eb-42df-bbd7-7f9566f2613a\") " Jul 07 07:05:02.181593 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.181484 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-maas-api-tls\") pod \"8b4c8b70-90eb-42df-bbd7-7f9566f2613a\" (UID: \"8b4c8b70-90eb-42df-bbd7-7f9566f2613a\") " Jul 07 07:05:02.181593 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.181569 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-openshift-service-ca\") pod \"8b4c8b70-90eb-42df-bbd7-7f9566f2613a\" (UID: \"8b4c8b70-90eb-42df-bbd7-7f9566f2613a\") " Jul 07 07:05:02.181923 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.181889 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8b4c8b70-90eb-42df-bbd7-7f9566f2613a" (UID: "8b4c8b70-90eb-42df-bbd7-7f9566f2613a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:05:02.183339 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.183320 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-kube-api-access-kshxc" (OuterVolumeSpecName: "kube-api-access-kshxc") pod "8b4c8b70-90eb-42df-bbd7-7f9566f2613a" (UID: "8b4c8b70-90eb-42df-bbd7-7f9566f2613a"). InnerVolumeSpecName "kube-api-access-kshxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:05:02.183495 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.183479 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8b4c8b70-90eb-42df-bbd7-7f9566f2613a" (UID: "8b4c8b70-90eb-42df-bbd7-7f9566f2613a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:05:02.282881 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.282848 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-openshift-service-ca\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:02.282881 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.282879 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kshxc\" (UniqueName: \"kubernetes.io/projected/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-kube-api-access-kshxc\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:02.283080 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.282892 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4c8b70-90eb-42df-bbd7-7f9566f2613a-maas-api-tls\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:02.877140 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.877104 2579 generic.go:356] "Generic (PLEG): container finished" podID="8b4c8b70-90eb-42df-bbd7-7f9566f2613a" containerID="13bfed68fbd6164ef97fa8ad964099f2add1a6a03eca3c20d51327e7a89db11e" exitCode=0 Jul 07 07:05:02.877596 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.877151 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" Jul 07 07:05:02.877596 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.877188 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" event={"ID":"8b4c8b70-90eb-42df-bbd7-7f9566f2613a","Type":"ContainerDied","Data":"13bfed68fbd6164ef97fa8ad964099f2add1a6a03eca3c20d51327e7a89db11e"} Jul 07 07:05:02.877596 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.877240 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8" event={"ID":"8b4c8b70-90eb-42df-bbd7-7f9566f2613a","Type":"ContainerDied","Data":"e267a57d40b91a9eb7720e57e65cc7aad823357e641be5fa8afe4b7041f589fe"} Jul 07 07:05:02.877596 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.877262 2579 scope.go:117] "RemoveContainer" containerID="13bfed68fbd6164ef97fa8ad964099f2add1a6a03eca3c20d51327e7a89db11e" Jul 07 07:05:02.886471 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.886452 2579 scope.go:117] "RemoveContainer" containerID="13bfed68fbd6164ef97fa8ad964099f2add1a6a03eca3c20d51327e7a89db11e" Jul 07 07:05:02.886707 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:05:02.886687 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13bfed68fbd6164ef97fa8ad964099f2add1a6a03eca3c20d51327e7a89db11e\": container with ID starting with 13bfed68fbd6164ef97fa8ad964099f2add1a6a03eca3c20d51327e7a89db11e not found: ID does not exist" containerID="13bfed68fbd6164ef97fa8ad964099f2add1a6a03eca3c20d51327e7a89db11e" Jul 07 07:05:02.886756 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.886716 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13bfed68fbd6164ef97fa8ad964099f2add1a6a03eca3c20d51327e7a89db11e"} err="failed to get container status \"13bfed68fbd6164ef97fa8ad964099f2add1a6a03eca3c20d51327e7a89db11e\": rpc error: code = NotFound desc = could not find container \"13bfed68fbd6164ef97fa8ad964099f2add1a6a03eca3c20d51327e7a89db11e\": container with ID starting with 13bfed68fbd6164ef97fa8ad964099f2add1a6a03eca3c20d51327e7a89db11e not found: ID does not exist" Jul 07 07:05:02.893793 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.893769 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8"] Jul 07 07:05:02.899746 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:02.899724 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-50d762-558f45446c-r6bs8"] Jul 07 07:05:04.784379 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:04.784335 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b4c8b70-90eb-42df-bbd7-7f9566f2613a" path="/var/lib/kubelet/pods/8b4c8b70-90eb-42df-bbd7-7f9566f2613a/volumes" Jul 07 07:05:06.065400 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.065315 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:05:06.219350 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.219311 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2qsh\" (UniqueName: \"kubernetes.io/projected/aa946518-5816-42af-9923-70d6e54de4e2-kube-api-access-v2qsh\") pod \"aa946518-5816-42af-9923-70d6e54de4e2\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " Jul 07 07:05:06.219581 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.219394 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-workload-socket\") pod \"aa946518-5816-42af-9923-70d6e54de4e2\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " Jul 07 07:05:06.219581 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.219438 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-istio-envoy\") pod \"aa946518-5816-42af-9923-70d6e54de4e2\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " Jul 07 07:05:06.219581 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.219462 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa946518-5816-42af-9923-70d6e54de4e2-istio-podinfo\") pod \"aa946518-5816-42af-9923-70d6e54de4e2\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " Jul 07 07:05:06.219581 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.219485 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-workload-certs\") pod \"aa946518-5816-42af-9923-70d6e54de4e2\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " Jul 07 07:05:06.219789 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.219616 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa946518-5816-42af-9923-70d6e54de4e2-istiod-ca-cert\") pod \"aa946518-5816-42af-9923-70d6e54de4e2\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " Jul 07 07:05:06.219789 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.219707 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-istio-data\") pod \"aa946518-5816-42af-9923-70d6e54de4e2\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " Jul 07 07:05:06.219789 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.219741 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa946518-5816-42af-9923-70d6e54de4e2-istio-token\") pod \"aa946518-5816-42af-9923-70d6e54de4e2\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " Jul 07 07:05:06.219789 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.219765 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-credential-socket\") pod \"aa946518-5816-42af-9923-70d6e54de4e2\" (UID: \"aa946518-5816-42af-9923-70d6e54de4e2\") " Jul 07 07:05:06.219789 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.219747 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aa946518-5816-42af-9923-70d6e54de4e2" (UID: "aa946518-5816-42af-9923-70d6e54de4e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:05:06.219789 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.219784 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aa946518-5816-42af-9923-70d6e54de4e2" (UID: "aa946518-5816-42af-9923-70d6e54de4e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:05:06.220095 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.219997 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa946518-5816-42af-9923-70d6e54de4e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aa946518-5816-42af-9923-70d6e54de4e2" (UID: "aa946518-5816-42af-9923-70d6e54de4e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:05:06.220149 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.220124 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-workload-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:06.220248 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.220214 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aa946518-5816-42af-9923-70d6e54de4e2" (UID: "aa946518-5816-42af-9923-70d6e54de4e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:05:06.220248 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.220143 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-workload-certs\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:06.220386 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.220259 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa946518-5816-42af-9923-70d6e54de4e2-istiod-ca-cert\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:06.220482 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.220455 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "aa946518-5816-42af-9923-70d6e54de4e2" (UID: "aa946518-5816-42af-9923-70d6e54de4e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:05:06.221648 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.221624 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aa946518-5816-42af-9923-70d6e54de4e2" (UID: "aa946518-5816-42af-9923-70d6e54de4e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:05:06.221776 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.221710 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa946518-5816-42af-9923-70d6e54de4e2-kube-api-access-v2qsh" (OuterVolumeSpecName: "kube-api-access-v2qsh") pod "aa946518-5816-42af-9923-70d6e54de4e2" (UID: "aa946518-5816-42af-9923-70d6e54de4e2"). InnerVolumeSpecName "kube-api-access-v2qsh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:05:06.222038 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.222020 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa946518-5816-42af-9923-70d6e54de4e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "aa946518-5816-42af-9923-70d6e54de4e2" (UID: "aa946518-5816-42af-9923-70d6e54de4e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:05:06.222096 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.222035 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa946518-5816-42af-9923-70d6e54de4e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aa946518-5816-42af-9923-70d6e54de4e2" (UID: "aa946518-5816-42af-9923-70d6e54de4e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 07:05:06.320868 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.320785 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-istio-data\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:06.320868 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.320814 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa946518-5816-42af-9923-70d6e54de4e2-istio-token\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:06.320868 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.320822 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-credential-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:06.320868 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.320832 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2qsh\" (UniqueName: \"kubernetes.io/projected/aa946518-5816-42af-9923-70d6e54de4e2-kube-api-access-v2qsh\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:06.320868 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.320842 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa946518-5816-42af-9923-70d6e54de4e2-istio-envoy\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:06.320868 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.320852 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa946518-5816-42af-9923-70d6e54de4e2-istio-podinfo\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:06.904278 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.904247 2579 generic.go:356] "Generic (PLEG): container finished" podID="aa946518-5816-42af-9923-70d6e54de4e2" containerID="793e8fa5486051a83e188d2f9521a58f23a25190337160cb97132fa4953a22b2" exitCode=0 Jul 07 07:05:06.904456 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.904295 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" Jul 07 07:05:06.904456 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.904317 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" event={"ID":"aa946518-5816-42af-9923-70d6e54de4e2","Type":"ContainerDied","Data":"793e8fa5486051a83e188d2f9521a58f23a25190337160cb97132fa4953a22b2"} Jul 07 07:05:06.904456 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.904379 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh" event={"ID":"aa946518-5816-42af-9923-70d6e54de4e2","Type":"ContainerDied","Data":"cce2d6947e5b2e6ccb5e28a37c11fa733b6d0ca43e3fe7e54ab018f2b762eb8f"} Jul 07 07:05:06.904456 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.904405 2579 scope.go:117] "RemoveContainer" containerID="793e8fa5486051a83e188d2f9521a58f23a25190337160cb97132fa4953a22b2" Jul 07 07:05:06.913246 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.913228 2579 scope.go:117] "RemoveContainer" containerID="a272d98791dc63c9e9694dbf3c8a087a8ef136bb25ef90adb7b3fbc33f7a0c3a" Jul 07 07:05:06.921530 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.921313 2579 scope.go:117] "RemoveContainer" containerID="793e8fa5486051a83e188d2f9521a58f23a25190337160cb97132fa4953a22b2" Jul 07 07:05:06.921667 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:05:06.921649 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"793e8fa5486051a83e188d2f9521a58f23a25190337160cb97132fa4953a22b2\": container with ID starting with 793e8fa5486051a83e188d2f9521a58f23a25190337160cb97132fa4953a22b2 not found: ID does not exist" containerID="793e8fa5486051a83e188d2f9521a58f23a25190337160cb97132fa4953a22b2" Jul 07 07:05:06.921709 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.921674 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"793e8fa5486051a83e188d2f9521a58f23a25190337160cb97132fa4953a22b2"} err="failed to get container status \"793e8fa5486051a83e188d2f9521a58f23a25190337160cb97132fa4953a22b2\": rpc error: code = NotFound desc = could not find container \"793e8fa5486051a83e188d2f9521a58f23a25190337160cb97132fa4953a22b2\": container with ID starting with 793e8fa5486051a83e188d2f9521a58f23a25190337160cb97132fa4953a22b2 not found: ID does not exist" Jul 07 07:05:06.921709 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.921690 2579 scope.go:117] "RemoveContainer" containerID="a272d98791dc63c9e9694dbf3c8a087a8ef136bb25ef90adb7b3fbc33f7a0c3a" Jul 07 07:05:06.921904 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:05:06.921891 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a272d98791dc63c9e9694dbf3c8a087a8ef136bb25ef90adb7b3fbc33f7a0c3a\": container with ID starting with a272d98791dc63c9e9694dbf3c8a087a8ef136bb25ef90adb7b3fbc33f7a0c3a not found: ID does not exist" containerID="a272d98791dc63c9e9694dbf3c8a087a8ef136bb25ef90adb7b3fbc33f7a0c3a" Jul 07 07:05:06.921946 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.921907 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a272d98791dc63c9e9694dbf3c8a087a8ef136bb25ef90adb7b3fbc33f7a0c3a"} err="failed to get container status \"a272d98791dc63c9e9694dbf3c8a087a8ef136bb25ef90adb7b3fbc33f7a0c3a\": rpc error: code = NotFound desc = could not find container \"a272d98791dc63c9e9694dbf3c8a087a8ef136bb25ef90adb7b3fbc33f7a0c3a\": container with ID starting with a272d98791dc63c9e9694dbf3c8a087a8ef136bb25ef90adb7b3fbc33f7a0c3a not found: ID does not exist" Jul 07 07:05:06.938136 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.938107 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh"] Jul 07 07:05:06.946324 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:06.946304 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-98f550-openshift-default-78584b5c9c-l8clh"] Jul 07 07:05:07.742805 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:07.742773 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:07.913580 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:07.913546 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:08.003458 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.003380 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k"] Jul 07 07:05:08.003633 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.003609 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" podUID="f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" containerName="istio-proxy" containerID="cri-o://4d93053f97e01a986c463358376a7948a830e0b4df65615a752b1886683c78eb" gracePeriod=30 Jul 07 07:05:08.014026 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.014001 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:08.232130 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.232098 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:08.582010 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.581952 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:08.783263 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.783231 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa946518-5816-42af-9923-70d6e54de4e2" path="/var/lib/kubelet/pods/aa946518-5816-42af-9923-70d6e54de4e2/volumes" Jul 07 07:05:08.938057 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.937982 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2"] Jul 07 07:05:08.938337 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.938325 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4ee35a1-562c-4867-8560-0ab0b88b5490" containerName="maas-api" Jul 07 07:05:08.938400 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.938338 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4ee35a1-562c-4867-8560-0ab0b88b5490" containerName="maas-api" Jul 07 07:05:08.938400 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.938351 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b4c8b70-90eb-42df-bbd7-7f9566f2613a" containerName="maas-api" Jul 07 07:05:08.938400 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.938356 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b4c8b70-90eb-42df-bbd7-7f9566f2613a" containerName="maas-api" Jul 07 07:05:08.938400 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.938379 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa946518-5816-42af-9923-70d6e54de4e2" containerName="istio-proxy" Jul 07 07:05:08.938400 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.938386 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa946518-5816-42af-9923-70d6e54de4e2" containerName="istio-proxy" Jul 07 07:05:08.938573 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.938451 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b4c8b70-90eb-42df-bbd7-7f9566f2613a" containerName="maas-api" Jul 07 07:05:08.938573 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.938461 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa946518-5816-42af-9923-70d6e54de4e2" containerName="istio-proxy" Jul 07 07:05:08.938573 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.938467 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4ee35a1-562c-4867-8560-0ab0b88b5490" containerName="maas-api" Jul 07 07:05:08.938573 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.938473 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa946518-5816-42af-9923-70d6e54de4e2" containerName="istio-proxy" Jul 07 07:05:08.940340 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.940324 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:08.943276 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.943255 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-94448f-openshift-default-dockercfg-r4d7j\"" Jul 07 07:05:08.953728 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.953708 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2"] Jul 07 07:05:08.963586 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:08.963562 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:09.046421 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.046387 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2008c626-76fe-41b5-9546-ba535db303f5-istiod-ca-cert\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.046605 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.046435 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-istio-data\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.046605 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.046465 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-workload-socket\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.046605 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.046492 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-credential-socket\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.046739 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.046594 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2008c626-76fe-41b5-9546-ba535db303f5-istio-token\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.046739 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.046676 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-workload-certs\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.046739 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.046702 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2008c626-76fe-41b5-9546-ba535db303f5-istio-podinfo\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.046739 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.046728 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-istio-envoy\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.046884 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.046754 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xstt9\" (UniqueName: \"kubernetes.io/projected/2008c626-76fe-41b5-9546-ba535db303f5-kube-api-access-xstt9\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.147764 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.147725 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-workload-certs\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.147951 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.147771 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2008c626-76fe-41b5-9546-ba535db303f5-istio-podinfo\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.147951 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.147804 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-istio-envoy\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.147951 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.147838 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xstt9\" (UniqueName: \"kubernetes.io/projected/2008c626-76fe-41b5-9546-ba535db303f5-kube-api-access-xstt9\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.147951 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.147913 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2008c626-76fe-41b5-9546-ba535db303f5-istiod-ca-cert\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.147951 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.147943 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-istio-data\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.148213 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.147973 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-workload-socket\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.148213 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.148004 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-credential-socket\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.148213 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.148126 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-workload-certs\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.148213 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.148177 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2008c626-76fe-41b5-9546-ba535db303f5-istio-token\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.148465 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.148298 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-workload-socket\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.148465 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.148392 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-credential-socket\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.148838 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.148813 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2008c626-76fe-41b5-9546-ba535db303f5-istiod-ca-cert\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.148887 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.148872 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-istio-data\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.150080 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.150055 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-istio-envoy\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.150286 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.150272 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2008c626-76fe-41b5-9546-ba535db303f5-istio-podinfo\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.156524 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.156498 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2008c626-76fe-41b5-9546-ba535db303f5-istio-token\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.156860 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.156843 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xstt9\" (UniqueName: \"kubernetes.io/projected/2008c626-76fe-41b5-9546-ba535db303f5-kube-api-access-xstt9\") pod \"e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.249957 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.249882 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:09.372009 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.371965 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:09.387736 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.387710 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2"] Jul 07 07:05:09.388913 ip-10-0-130-176 kubenswrapper[2579]: W0707 07:05:09.388887 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2008c626_76fe_41b5_9546_ba535db303f5.slice/crio-494c78b0830b9f06f9b63973f9ecaa8af52907741df396c4da0787b1ef2eeca1 WatchSource:0}: Error finding container 494c78b0830b9f06f9b63973f9ecaa8af52907741df396c4da0787b1ef2eeca1: Status 404 returned error can't find the container with id 494c78b0830b9f06f9b63973f9ecaa8af52907741df396c4da0787b1ef2eeca1 Jul 07 07:05:09.391000 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.390970 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:05:09.391061 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.391029 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:05:09.391061 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.391056 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:05:09.923997 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.923966 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" event={"ID":"2008c626-76fe-41b5-9546-ba535db303f5","Type":"ContainerStarted","Data":"d8cc6b1751c6c316e7362f66b279f3f09dfefae802473879cb204054d8072dee"} Jul 07 07:05:09.924430 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.924013 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" event={"ID":"2008c626-76fe-41b5-9546-ba535db303f5","Type":"ContainerStarted","Data":"494c78b0830b9f06f9b63973f9ecaa8af52907741df396c4da0787b1ef2eeca1"} Jul 07 07:05:09.948704 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:09.948656 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" podStartSLOduration=1.94864126 podStartE2EDuration="1.94864126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:05:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:05:09.947652403 +0000 UTC m=+1985.787493398" watchObservedRunningTime="2026-07-07 07:05:09.94864126 +0000 UTC m=+1985.788482227" Jul 07 07:05:10.250570 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:10.250492 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:10.255316 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:10.255289 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:10.929430 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:10.929400 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:10.930345 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:10.930324 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:11.933293 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:11.933250 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:11.996213 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:11.996178 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t"] Jul 07 07:05:11.996577 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:11.996559 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa946518-5816-42af-9923-70d6e54de4e2" containerName="istio-proxy" Jul 07 07:05:11.996577 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:11.996575 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa946518-5816-42af-9923-70d6e54de4e2" containerName="istio-proxy" Jul 07 07:05:11.998690 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:11.998673 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" Jul 07 07:05:12.002974 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.002953 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-94448f-serving-cert\"" Jul 07 07:05:12.002974 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.002966 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-754v8\"" Jul 07 07:05:12.013072 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.013050 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t"] Jul 07 07:05:12.025154 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.025131 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:12.105523 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.105484 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:12.175615 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.175586 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw2lc\" (UniqueName: \"kubernetes.io/projected/58de2004-cf1b-45e1-9d36-cf1965ed5676-kube-api-access-xw2lc\") pod \"maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t\" (UID: \"58de2004-cf1b-45e1-9d36-cf1965ed5676\") " pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" Jul 07 07:05:12.175834 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.175713 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58de2004-cf1b-45e1-9d36-cf1965ed5676-maas-api-tls\") pod \"maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t\" (UID: \"58de2004-cf1b-45e1-9d36-cf1965ed5676\") " pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" Jul 07 07:05:12.175834 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.175773 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58de2004-cf1b-45e1-9d36-cf1965ed5676-openshift-service-ca\") pod \"maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t\" (UID: \"58de2004-cf1b-45e1-9d36-cf1965ed5676\") " pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" Jul 07 07:05:12.215216 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.215129 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:12.277228 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.277197 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58de2004-cf1b-45e1-9d36-cf1965ed5676-maas-api-tls\") pod \"maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t\" (UID: \"58de2004-cf1b-45e1-9d36-cf1965ed5676\") " pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" Jul 07 07:05:12.277409 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.277235 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58de2004-cf1b-45e1-9d36-cf1965ed5676-openshift-service-ca\") pod \"maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t\" (UID: \"58de2004-cf1b-45e1-9d36-cf1965ed5676\") " pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" Jul 07 07:05:12.277409 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.277275 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw2lc\" (UniqueName: \"kubernetes.io/projected/58de2004-cf1b-45e1-9d36-cf1965ed5676-kube-api-access-xw2lc\") pod \"maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t\" (UID: \"58de2004-cf1b-45e1-9d36-cf1965ed5676\") " pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" Jul 07 07:05:12.278002 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.277979 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58de2004-cf1b-45e1-9d36-cf1965ed5676-openshift-service-ca\") pod \"maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t\" (UID: \"58de2004-cf1b-45e1-9d36-cf1965ed5676\") " pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" Jul 07 07:05:12.279676 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.279657 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58de2004-cf1b-45e1-9d36-cf1965ed5676-maas-api-tls\") pod \"maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t\" (UID: \"58de2004-cf1b-45e1-9d36-cf1965ed5676\") " pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" Jul 07 07:05:12.288415 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.288393 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw2lc\" (UniqueName: \"kubernetes.io/projected/58de2004-cf1b-45e1-9d36-cf1965ed5676-kube-api-access-xw2lc\") pod \"maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t\" (UID: \"58de2004-cf1b-45e1-9d36-cf1965ed5676\") " pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" Jul 07 07:05:12.309428 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.309352 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" Jul 07 07:05:12.446964 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.446931 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t"] Jul 07 07:05:12.447833 ip-10-0-130-176 kubenswrapper[2579]: W0707 07:05:12.447802 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58de2004_cf1b_45e1_9d36_cf1965ed5676.slice/crio-423da920008abaadbb6bdd5ce1d1706872fd0075df96c3c09dc816cb251ecb66 WatchSource:0}: Error finding container 423da920008abaadbb6bdd5ce1d1706872fd0075df96c3c09dc816cb251ecb66: Status 404 returned error can't find the container with id 423da920008abaadbb6bdd5ce1d1706872fd0075df96c3c09dc816cb251ecb66 Jul 07 07:05:12.466257 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.466103 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:12.570294 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.570268 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:12.942790 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.942756 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" event={"ID":"58de2004-cf1b-45e1-9d36-cf1965ed5676","Type":"ContainerStarted","Data":"3c56af0c7c8dcb092f4b4ad5483b72e4a5f4afc71ab25a3332baeaeb3b451096"} Jul 07 07:05:12.943150 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.942809 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" event={"ID":"58de2004-cf1b-45e1-9d36-cf1965ed5676","Type":"ContainerStarted","Data":"423da920008abaadbb6bdd5ce1d1706872fd0075df96c3c09dc816cb251ecb66"} Jul 07 07:05:12.977171 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:12.977120 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" podStartSLOduration=1.606167898 podStartE2EDuration="1.97710539s" totalImagesPullingTime="370.937492ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:05:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:05:12.974912254 +0000 UTC m=+1988.814753224" watchObservedRunningTime="2026-07-07 07:05:12.97710539 +0000 UTC m=+1988.816946357" Jul 07 07:05:13.245545 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.245519 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:05:13.387267 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.387179 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-workload-socket\") pod \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " Jul 07 07:05:13.387267 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.387213 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-data\") pod \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " Jul 07 07:05:13.387531 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.387274 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-envoy\") pod \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " Jul 07 07:05:13.387531 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.387289 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-workload-certs\") pod \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " Jul 07 07:05:13.387531 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.387315 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mplg4\" (UniqueName: \"kubernetes.io/projected/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-kube-api-access-mplg4\") pod \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " Jul 07 07:05:13.387531 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.387390 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-credential-socket\") pod \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " Jul 07 07:05:13.387531 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.387433 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-token\") pod \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " Jul 07 07:05:13.387531 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.387486 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istiod-ca-cert\") pod \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " Jul 07 07:05:13.387822 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.387524 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" (UID: "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:05:13.387822 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.387531 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-podinfo\") pod \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\" (UID: \"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab\") " Jul 07 07:05:13.387822 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.387679 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" (UID: "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:05:13.387987 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.387852 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" (UID: "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:05:13.387987 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.387968 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" (UID: "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:05:13.388089 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.388028 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-workload-certs\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.388089 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.388049 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-credential-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.388089 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.388063 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-workload-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.388208 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.388185 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-data" (OuterVolumeSpecName: "istio-data") pod "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" (UID: "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:05:13.389648 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.389618 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" (UID: "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:05:13.389648 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.389635 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-token" (OuterVolumeSpecName: "istio-token") pod "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" (UID: "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:05:13.389789 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.389673 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-kube-api-access-mplg4" (OuterVolumeSpecName: "kube-api-access-mplg4") pod "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" (UID: "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab"). InnerVolumeSpecName "kube-api-access-mplg4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:05:13.389989 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.389976 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" (UID: "f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 07:05:13.488763 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.488728 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-token\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.488763 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.488757 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istiod-ca-cert\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.488936 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.488770 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-podinfo\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.488936 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.488783 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-data\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.488936 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.488794 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-istio-envoy\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.488936 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.488806 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mplg4\" (UniqueName: \"kubernetes.io/projected/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab-kube-api-access-mplg4\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:05:13.949492 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.949460 2579 generic.go:356] "Generic (PLEG): container finished" podID="f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" containerID="4d93053f97e01a986c463358376a7948a830e0b4df65615a752b1886683c78eb" exitCode=0 Jul 07 07:05:13.949925 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.949513 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" Jul 07 07:05:13.949925 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.949537 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" event={"ID":"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab","Type":"ContainerDied","Data":"4d93053f97e01a986c463358376a7948a830e0b4df65615a752b1886683c78eb"} Jul 07 07:05:13.949925 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.949589 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k" event={"ID":"f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab","Type":"ContainerDied","Data":"b6e02d3f4174780e5e3ca35f2eeae88cf506296f583d06755eec233b1737106c"} Jul 07 07:05:13.949925 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.949610 2579 scope.go:117] "RemoveContainer" containerID="4d93053f97e01a986c463358376a7948a830e0b4df65615a752b1886683c78eb" Jul 07 07:05:13.950156 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.949924 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" Jul 07 07:05:13.959235 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.959219 2579 scope.go:117] "RemoveContainer" containerID="b69f2f1e1fbf0c2488182ddf5870f9db959482ec1bbf520b880f374459159876" Jul 07 07:05:13.966766 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.966749 2579 scope.go:117] "RemoveContainer" containerID="4d93053f97e01a986c463358376a7948a830e0b4df65615a752b1886683c78eb" Jul 07 07:05:13.967003 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:05:13.966983 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d93053f97e01a986c463358376a7948a830e0b4df65615a752b1886683c78eb\": container with ID starting with 4d93053f97e01a986c463358376a7948a830e0b4df65615a752b1886683c78eb not found: ID does not exist" containerID="4d93053f97e01a986c463358376a7948a830e0b4df65615a752b1886683c78eb" Jul 07 07:05:13.967085 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.967010 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d93053f97e01a986c463358376a7948a830e0b4df65615a752b1886683c78eb"} err="failed to get container status \"4d93053f97e01a986c463358376a7948a830e0b4df65615a752b1886683c78eb\": rpc error: code = NotFound desc = could not find container \"4d93053f97e01a986c463358376a7948a830e0b4df65615a752b1886683c78eb\": container with ID starting with 4d93053f97e01a986c463358376a7948a830e0b4df65615a752b1886683c78eb not found: ID does not exist" Jul 07 07:05:13.967085 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.967025 2579 scope.go:117] "RemoveContainer" containerID="b69f2f1e1fbf0c2488182ddf5870f9db959482ec1bbf520b880f374459159876" Jul 07 07:05:13.967252 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:05:13.967237 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b69f2f1e1fbf0c2488182ddf5870f9db959482ec1bbf520b880f374459159876\": container with ID starting with b69f2f1e1fbf0c2488182ddf5870f9db959482ec1bbf520b880f374459159876 not found: ID does not exist" containerID="b69f2f1e1fbf0c2488182ddf5870f9db959482ec1bbf520b880f374459159876" Jul 07 07:05:13.967291 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.967255 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b69f2f1e1fbf0c2488182ddf5870f9db959482ec1bbf520b880f374459159876"} err="failed to get container status \"b69f2f1e1fbf0c2488182ddf5870f9db959482ec1bbf520b880f374459159876\": rpc error: code = NotFound desc = could not find container \"b69f2f1e1fbf0c2488182ddf5870f9db959482ec1bbf520b880f374459159876\": container with ID starting with b69f2f1e1fbf0c2488182ddf5870f9db959482ec1bbf520b880f374459159876 not found: ID does not exist" Jul 07 07:05:13.973232 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.973202 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k"] Jul 07 07:05:13.978719 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:13.978700 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-50d762-openshift-default-7dbdc5f7f7-lqq9k"] Jul 07 07:05:14.782987 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:14.782953 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" path="/var/lib/kubelet/pods/f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab/volumes" Jul 07 07:05:16.374296 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:16.374259 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:16.718431 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:16.718331 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:17.037387 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.033842 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59"] Jul 07 07:05:17.046387 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.038356 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" containerName="istio-proxy" Jul 07 07:05:17.046593 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.046574 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" containerName="istio-proxy" Jul 07 07:05:17.046777 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.046764 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" containerName="istio-proxy" Jul 07 07:05:17.046898 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.046887 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" containerName="istio-proxy" Jul 07 07:05:17.047232 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.047217 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" containerName="istio-proxy" Jul 07 07:05:17.050952 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.050909 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.054605 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.054578 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-c72c9a-openshift-default-dockercfg-xsxx7\"" Jul 07 07:05:17.056540 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.056502 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59"] Jul 07 07:05:17.062537 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.062512 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:17.225210 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.225171 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-credential-socket\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.225442 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.225232 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z5ns\" (UniqueName: \"kubernetes.io/projected/a94128ba-f6be-4046-8bc6-8bf726881ef0-kube-api-access-9z5ns\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.225442 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.225282 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-envoy\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.225442 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.225378 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-workload-certs\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.225442 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.225433 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-workload-socket\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.225629 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.225463 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-podinfo\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.225629 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.225485 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-token\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.225629 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.225513 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a94128ba-f6be-4046-8bc6-8bf726881ef0-istiod-ca-cert\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.225629 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.225535 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-data\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.326934 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.326843 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-workload-socket\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.326934 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.326883 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-podinfo\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.326934 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.326906 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-token\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.326934 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.326931 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a94128ba-f6be-4046-8bc6-8bf726881ef0-istiod-ca-cert\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.327277 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.326951 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-data\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.327277 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.326968 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-credential-socket\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.327277 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.327000 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z5ns\" (UniqueName: \"kubernetes.io/projected/a94128ba-f6be-4046-8bc6-8bf726881ef0-kube-api-access-9z5ns\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.327277 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.327047 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-envoy\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.327277 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.327106 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-workload-certs\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.327566 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.327286 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-workload-socket\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.327566 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.327319 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-data\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.327566 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.327475 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-credential-socket\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.327566 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.327488 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-workload-certs\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.327786 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.327767 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a94128ba-f6be-4046-8bc6-8bf726881ef0-istiod-ca-cert\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.337904 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.337878 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-podinfo\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.338583 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.338557 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-envoy\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.342789 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.342760 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z5ns\" (UniqueName: \"kubernetes.io/projected/a94128ba-f6be-4046-8bc6-8bf726881ef0-kube-api-access-9z5ns\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.346384 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.343163 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-token\") pod \"e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.365808 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.365775 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:17.523383 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.523338 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59"] Jul 07 07:05:17.525534 ip-10-0-130-176 kubenswrapper[2579]: W0707 07:05:17.525508 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda94128ba_f6be_4046_8bc6_8bf726881ef0.slice/crio-f4519ebd8774ce7641bf41bcce8aa10175f6dca91f6d043335a0ab0a0f97f585 WatchSource:0}: Error finding container f4519ebd8774ce7641bf41bcce8aa10175f6dca91f6d043335a0ab0a0f97f585: Status 404 returned error can't find the container with id f4519ebd8774ce7641bf41bcce8aa10175f6dca91f6d043335a0ab0a0f97f585 Jul 07 07:05:17.527689 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.527657 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:05:17.527773 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.527717 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:05:17.527773 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.527745 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:05:17.786375 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.786335 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:17.974864 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.974777 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" event={"ID":"a94128ba-f6be-4046-8bc6-8bf726881ef0","Type":"ContainerStarted","Data":"3fe6748c62578baa1f77e0b46d14cdea24c2ef8f1b28fd5575b435c594096533"} Jul 07 07:05:17.974864 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:17.974832 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" event={"ID":"a94128ba-f6be-4046-8bc6-8bf726881ef0","Type":"ContainerStarted","Data":"f4519ebd8774ce7641bf41bcce8aa10175f6dca91f6d043335a0ab0a0f97f585"} Jul 07 07:05:18.005212 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:18.005165 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" podStartSLOduration=2.005149364 podStartE2EDuration="2.005149364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:05:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:05:18.00488232 +0000 UTC m=+1993.844723288" watchObservedRunningTime="2026-07-07 07:05:18.005149364 +0000 UTC m=+1993.844990331" Jul 07 07:05:18.366506 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:18.366473 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:18.368268 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:18.368238 2579 patch_prober.go:28] interesting pod/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 07 07:05:18.368534 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:18.368508 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" podUID="a94128ba-f6be-4046-8bc6-8bf726881ef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 07 07:05:19.370647 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:19.370621 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:19.962868 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:19.962836 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" Jul 07 07:05:19.974632 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:19.974600 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:19.984851 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:19.984829 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:19.985841 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:19.985823 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:20.065457 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.065424 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7"] Jul 07 07:05:20.066619 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.066576 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1ead2c4-dd28-4349-a5b9-4fa6cad7dcab" containerName="istio-proxy" Jul 07 07:05:20.069655 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.069391 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:05:20.071386 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.071302 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwkvq\" (UniqueName: \"kubernetes.io/projected/97f050a1-5065-456f-a8d4-16dff3c9f2b4-kube-api-access-vwkvq\") pod \"maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7\" (UID: \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\") " pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:05:20.071517 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.071437 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97f050a1-5065-456f-a8d4-16dff3c9f2b4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7\" (UID: \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\") " pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:05:20.071517 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.071478 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97f050a1-5065-456f-a8d4-16dff3c9f2b4-maas-api-tls\") pod \"maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7\" (UID: \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\") " pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:05:20.073120 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.073098 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-c72c9a-serving-cert\"" Jul 07 07:05:20.081695 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.081660 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7"] Jul 07 07:05:20.083527 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.083504 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:20.172432 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.172356 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwkvq\" (UniqueName: \"kubernetes.io/projected/97f050a1-5065-456f-a8d4-16dff3c9f2b4-kube-api-access-vwkvq\") pod \"maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7\" (UID: \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\") " pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:05:20.173140 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.173108 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97f050a1-5065-456f-a8d4-16dff3c9f2b4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7\" (UID: \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\") " pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:05:20.173341 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.173324 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97f050a1-5065-456f-a8d4-16dff3c9f2b4-maas-api-tls\") pod \"maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7\" (UID: \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\") " pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:05:20.173640 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:05:20.173550 2579 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-c72c9a-serving-cert: secret "maas-api-e2e-shared-b-c72c9a-serving-cert" not found Jul 07 07:05:20.173803 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:05:20.173792 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f050a1-5065-456f-a8d4-16dff3c9f2b4-maas-api-tls podName:97f050a1-5065-456f-a8d4-16dff3c9f2b4 nodeName:}" failed. No retries permitted until 2026-07-07 07:05:20.673771109 +0000 UTC m=+1996.513612071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/97f050a1-5065-456f-a8d4-16dff3c9f2b4-maas-api-tls") pod "maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" (UID: "97f050a1-5065-456f-a8d4-16dff3c9f2b4") : secret "maas-api-e2e-shared-b-c72c9a-serving-cert" not found Jul 07 07:05:20.174133 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.173942 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97f050a1-5065-456f-a8d4-16dff3c9f2b4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7\" (UID: \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\") " pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:05:20.180766 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.180736 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:20.185904 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.185018 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwkvq\" (UniqueName: \"kubernetes.io/projected/97f050a1-5065-456f-a8d4-16dff3c9f2b4-kube-api-access-vwkvq\") pod \"maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7\" (UID: \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\") " pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:05:20.471352 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.471301 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:20.678072 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.678040 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97f050a1-5065-456f-a8d4-16dff3c9f2b4-maas-api-tls\") pod \"maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7\" (UID: \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\") " pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:05:20.680874 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.680841 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97f050a1-5065-456f-a8d4-16dff3c9f2b4-maas-api-tls\") pod \"maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7\" (UID: \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\") " pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:05:20.686564 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.686542 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:05:20.866491 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.866444 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7"] Jul 07 07:05:20.868124 ip-10-0-130-176 kubenswrapper[2579]: W0707 07:05:20.868097 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97f050a1_5065_456f_a8d4_16dff3c9f2b4.slice/crio-caf27b1b645a07ae51f779dfe6c3701b6fe622196201cb18466663fa61ff6ea0 WatchSource:0}: Error finding container caf27b1b645a07ae51f779dfe6c3701b6fe622196201cb18466663fa61ff6ea0: Status 404 returned error can't find the container with id caf27b1b645a07ae51f779dfe6c3701b6fe622196201cb18466663fa61ff6ea0 Jul 07 07:05:20.886918 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.886889 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:20.991680 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:20.991594 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" event={"ID":"97f050a1-5065-456f-a8d4-16dff3c9f2b4","Type":"ContainerStarted","Data":"caf27b1b645a07ae51f779dfe6c3701b6fe622196201cb18466663fa61ff6ea0"} Jul 07 07:05:21.034716 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:21.034677 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:21.999634 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:21.999595 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" event={"ID":"97f050a1-5065-456f-a8d4-16dff3c9f2b4","Type":"ContainerStarted","Data":"93b4ebac0e43a586e96b4c8100b923e860eb936f8642a36cb1e8caa4a5ac5221"} Jul 07 07:05:22.000102 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:21.999910 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:05:22.045946 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:22.045889 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" podStartSLOduration=1.5318342299999999 podStartE2EDuration="2.045871899s" totalImagesPullingTime="514.037669ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:05:22.043412527 +0000 UTC m=+1997.883253502" watchObservedRunningTime="2026-07-07 07:05:22.045871899 +0000 UTC m=+1997.885712866" Jul 07 07:05:24.011437 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:24.011400 2579 generic.go:356] "Generic (PLEG): container finished" podID="2008c626-76fe-41b5-9546-ba535db303f5" containerID="d8cc6b1751c6c316e7362f66b279f3f09dfefae802473879cb204054d8072dee" exitCode=0 Jul 07 07:05:24.011856 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:24.011460 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" event={"ID":"2008c626-76fe-41b5-9546-ba535db303f5","Type":"ContainerDied","Data":"d8cc6b1751c6c316e7362f66b279f3f09dfefae802473879cb204054d8072dee"} Jul 07 07:05:24.011856 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:24.011810 2579 scope.go:117] "RemoveContainer" containerID="d8cc6b1751c6c316e7362f66b279f3f09dfefae802473879cb204054d8072dee" Jul 07 07:05:24.012801 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:24.012774 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:05:24.012869 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:24.012843 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:05:24.012911 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:24.012873 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:05:24.488339 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:24.488304 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:25.018990 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:25.018959 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" event={"ID":"2008c626-76fe-41b5-9546-ba535db303f5","Type":"ContainerStarted","Data":"584db76973b7c9340f2e38e46c3e0691853ea116821604b8733fee3786361621"} Jul 07 07:05:25.250145 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:25.250100 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:25.251982 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:25.251478 2579 patch_prober.go:28] interesting pod/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 07 07:05:25.251982 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:25.251532 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" podUID="2008c626-76fe-41b5-9546-ba535db303f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 07 07:05:25.930013 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:25.929954 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:26.250947 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:26.250847 2579 patch_prober.go:28] interesting pod/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 07 07:05:26.251544 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:26.251510 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" podUID="2008c626-76fe-41b5-9546-ba535db303f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 07 07:05:27.251158 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:27.251114 2579 patch_prober.go:28] interesting pod/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 07 07:05:27.251684 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:27.251181 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" podUID="2008c626-76fe-41b5-9546-ba535db303f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 07 07:05:28.010465 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:28.010434 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:05:28.251291 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:28.251255 2579 patch_prober.go:28] interesting pod/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 07 07:05:28.252306 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:28.252272 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" podUID="2008c626-76fe-41b5-9546-ba535db303f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 07 07:05:29.251191 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:29.251151 2579 patch_prober.go:28] interesting pod/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 07 07:05:29.252270 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:29.252234 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" podUID="2008c626-76fe-41b5-9546-ba535db303f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 07 07:05:30.256513 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:30.256473 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:30.257570 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:30.257546 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:05:33.148212 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:33.148179 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:33.255410 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:33.255355 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:35.385011 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:35.384966 2579 patch_prober.go:28] interesting pod/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:47628->10.134.0.54:15021: read: connection reset by peer" start-of-body= Jul 07 07:05:35.385455 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:35.385036 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" podUID="a94128ba-f6be-4046-8bc6-8bf726881ef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:47628->10.134.0.54:15021: read: connection reset by peer" Jul 07 07:05:36.093211 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:36.093177 2579 generic.go:356] "Generic (PLEG): container finished" podID="a94128ba-f6be-4046-8bc6-8bf726881ef0" containerID="3fe6748c62578baa1f77e0b46d14cdea24c2ef8f1b28fd5575b435c594096533" exitCode=0 Jul 07 07:05:36.093416 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:36.093251 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" event={"ID":"a94128ba-f6be-4046-8bc6-8bf726881ef0","Type":"ContainerDied","Data":"3fe6748c62578baa1f77e0b46d14cdea24c2ef8f1b28fd5575b435c594096533"} Jul 07 07:05:36.093686 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:36.093671 2579 scope.go:117] "RemoveContainer" containerID="3fe6748c62578baa1f77e0b46d14cdea24c2ef8f1b28fd5575b435c594096533" Jul 07 07:05:36.094725 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:36.094699 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:05:36.094806 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:36.094791 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:05:36.094844 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:36.094835 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 07 07:05:37.100505 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:37.100462 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" event={"ID":"a94128ba-f6be-4046-8bc6-8bf726881ef0","Type":"ContainerStarted","Data":"eedb9ed673ecdc7868436cfb3ecf0b42b566d0be515c679bf7ba73a315b04d41"} Jul 07 07:05:37.366671 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:37.366584 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:37.366834 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:37.366807 2579 patch_prober.go:28] interesting pod/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 07 07:05:37.366881 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:37.366852 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" podUID="a94128ba-f6be-4046-8bc6-8bf726881ef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 07 07:05:38.366847 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:38.366807 2579 patch_prober.go:28] interesting pod/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 07 07:05:38.367244 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:38.366875 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" podUID="a94128ba-f6be-4046-8bc6-8bf726881ef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 07 07:05:39.366502 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:39.366465 2579 patch_prober.go:28] interesting pod/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 07 07:05:39.366698 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:39.366521 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" podUID="a94128ba-f6be-4046-8bc6-8bf726881ef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 07 07:05:40.366548 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:40.366502 2579 patch_prober.go:28] interesting pod/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 07 07:05:40.367038 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:40.366578 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" podUID="a94128ba-f6be-4046-8bc6-8bf726881ef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 07 07:05:41.369919 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:41.369887 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:41.370407 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:41.370279 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:41.370860 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:41.370839 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:05:41.883989 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:41.883951 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:41.988141 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:41.988104 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:05:44.894612 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:05:44.894581 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:30.545771 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:06:30.545679 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:06:50.864701 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:06:50.864659 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:07:04.853469 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:07:04.853438 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 07:07:04.857614 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:07:04.857594 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 07:08:26.416106 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:08:26.416073 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:26.502283 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:08:26.502250 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:32.818475 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:08:32.818440 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:33.653646 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:08:33.653604 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:08:56.160811 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:08:56.160778 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:09:46.514024 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:09:46.513975 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:09:46.612710 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:09:46.612632 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:09:53.971606 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:09:53.971569 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:10:02.001121 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:02.001085 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:10:28.166713 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:28.166673 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:10:28.941304 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:28.941264 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:10:33.904498 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:33.904458 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:10:34.083755 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:34.083713 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:10:36.798720 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:36.798688 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55"] Jul 07 07:10:36.802198 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:36.802180 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:36.805128 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:36.805108 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-94448f\"/\"kube-root-ca.crt\"" Jul 07 07:10:36.806164 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:36.806141 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-94448f\"/\"rate-test-model-94448f-kserve-self-signed-certs\"" Jul 07 07:10:36.806267 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:36.806170 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-94448f\"/\"openshift-service-ca.crt\"" Jul 07 07:10:36.806267 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:36.806182 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-94448f\"/\"default-dockercfg-hm5f9\"" Jul 07 07:10:36.821303 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:36.821280 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55"] Jul 07 07:10:36.829588 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:36.829565 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:10:36.919825 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:36.919794 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45zbd\" (UniqueName: \"kubernetes.io/projected/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-kube-api-access-45zbd\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:36.919972 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:36.919844 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-tmp-dir\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:36.919972 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:36.919898 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-dshm\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:36.919972 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:36.919924 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-home\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:36.919972 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:36.919955 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-model-cache\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:36.920121 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:36.919979 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-tls-certs\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:37.020607 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.020555 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45zbd\" (UniqueName: \"kubernetes.io/projected/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-kube-api-access-45zbd\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:37.020791 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.020646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-tmp-dir\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:37.020791 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.020703 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-dshm\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:37.020791 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.020743 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-home\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:37.020791 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.020779 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-model-cache\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:37.020974 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.020821 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-tls-certs\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:37.021092 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.021066 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-tmp-dir\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:37.021162 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.021121 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-model-cache\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:37.021219 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.021184 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-home\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:37.022900 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.022878 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-dshm\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:37.023201 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.023180 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-tls-certs\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:37.029665 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.029638 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45zbd\" (UniqueName: \"kubernetes.io/projected/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-kube-api-access-45zbd\") pod \"rate-test-model-94448f-kserve-848f44dc67-9dc55\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:37.111480 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.111412 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:37.247379 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.247342 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55"] Jul 07 07:10:37.249409 ip-10-0-130-176 kubenswrapper[2579]: W0707 07:10:37.249354 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba0c6ed3_5124_4a6b_97a3_ce157f1da5dc.slice/crio-ecc006c0837a5e5f4fdd3b73dbab1c935dc8bafaf33132706740b62e2b5ce850 WatchSource:0}: Error finding container ecc006c0837a5e5f4fdd3b73dbab1c935dc8bafaf33132706740b62e2b5ce850: Status 404 returned error can't find the container with id ecc006c0837a5e5f4fdd3b73dbab1c935dc8bafaf33132706740b62e2b5ce850 Jul 07 07:10:37.675166 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.675125 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" event={"ID":"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc","Type":"ContainerStarted","Data":"f02cedbc6e9883026f349f5722390dd8636426c0c53ae44e2a7c5abf1112364a"} Jul 07 07:10:37.675166 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.675172 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" event={"ID":"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc","Type":"ContainerStarted","Data":"ecc006c0837a5e5f4fdd3b73dbab1c935dc8bafaf33132706740b62e2b5ce850"} Jul 07 07:10:37.694127 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:37.694077 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" podStartSLOduration=1.694063146 podStartE2EDuration="1.694063146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:10:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:10:37.693059265 +0000 UTC m=+2313.532900232" watchObservedRunningTime="2026-07-07 07:10:37.694063146 +0000 UTC m=+2313.533904146" Jul 07 07:10:42.594722 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:42.594689 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:10:42.693496 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:42.693464 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:10:47.112098 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:47.112061 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:47.112098 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:47.112110 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:47.124304 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:47.124278 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:47.738482 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:47.738451 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:10:57.105401 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:10:57.105301 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:11:15.225940 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:11:15.225907 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:11:58.079046 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:11:58.079006 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55"] Jul 07 07:11:58.100688 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:11:58.100655 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:11:59.023182 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:11:59.023151 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:00.070942 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:00.070910 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:00.603146 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:00.603112 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:00.741602 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:00.741552 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:00.930174 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:00.930097 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t"] Jul 07 07:12:00.930385 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:00.930314 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" podUID="58de2004-cf1b-45e1-9d36-cf1965ed5676" containerName="maas-api" containerID="cri-o://3c56af0c7c8dcb092f4b4ad5483b72e4a5f4afc71ab25a3332baeaeb3b451096" gracePeriod=30 Jul 07 07:12:00.968494 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:00.968469 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:01.127281 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:01.127249 2579 generic.go:356] "Generic (PLEG): container finished" podID="58de2004-cf1b-45e1-9d36-cf1965ed5676" containerID="3c56af0c7c8dcb092f4b4ad5483b72e4a5f4afc71ab25a3332baeaeb3b451096" exitCode=0 Jul 07 07:12:01.127655 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:01.127321 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" event={"ID":"58de2004-cf1b-45e1-9d36-cf1965ed5676","Type":"ContainerDied","Data":"3c56af0c7c8dcb092f4b4ad5483b72e4a5f4afc71ab25a3332baeaeb3b451096"} Jul 07 07:12:01.182145 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:01.182091 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" Jul 07 07:12:01.303475 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:01.303449 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw2lc\" (UniqueName: \"kubernetes.io/projected/58de2004-cf1b-45e1-9d36-cf1965ed5676-kube-api-access-xw2lc\") pod \"58de2004-cf1b-45e1-9d36-cf1965ed5676\" (UID: \"58de2004-cf1b-45e1-9d36-cf1965ed5676\") " Jul 07 07:12:01.303649 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:01.303521 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58de2004-cf1b-45e1-9d36-cf1965ed5676-openshift-service-ca\") pod \"58de2004-cf1b-45e1-9d36-cf1965ed5676\" (UID: \"58de2004-cf1b-45e1-9d36-cf1965ed5676\") " Jul 07 07:12:01.303649 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:01.303593 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58de2004-cf1b-45e1-9d36-cf1965ed5676-maas-api-tls\") pod \"58de2004-cf1b-45e1-9d36-cf1965ed5676\" (UID: \"58de2004-cf1b-45e1-9d36-cf1965ed5676\") " Jul 07 07:12:01.303933 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:01.303896 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58de2004-cf1b-45e1-9d36-cf1965ed5676-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "58de2004-cf1b-45e1-9d36-cf1965ed5676" (UID: "58de2004-cf1b-45e1-9d36-cf1965ed5676"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:12:01.305636 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:01.305608 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58de2004-cf1b-45e1-9d36-cf1965ed5676-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "58de2004-cf1b-45e1-9d36-cf1965ed5676" (UID: "58de2004-cf1b-45e1-9d36-cf1965ed5676"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:12:01.305738 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:01.305689 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58de2004-cf1b-45e1-9d36-cf1965ed5676-kube-api-access-xw2lc" (OuterVolumeSpecName: "kube-api-access-xw2lc") pod "58de2004-cf1b-45e1-9d36-cf1965ed5676" (UID: "58de2004-cf1b-45e1-9d36-cf1965ed5676"). InnerVolumeSpecName "kube-api-access-xw2lc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:12:01.404427 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:01.404399 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58de2004-cf1b-45e1-9d36-cf1965ed5676-openshift-service-ca\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:01.404427 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:01.404424 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58de2004-cf1b-45e1-9d36-cf1965ed5676-maas-api-tls\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:01.404427 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:01.404433 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw2lc\" (UniqueName: \"kubernetes.io/projected/58de2004-cf1b-45e1-9d36-cf1965ed5676-kube-api-access-xw2lc\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:02.134101 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:02.134063 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" event={"ID":"58de2004-cf1b-45e1-9d36-cf1965ed5676","Type":"ContainerDied","Data":"423da920008abaadbb6bdd5ce1d1706872fd0075df96c3c09dc816cb251ecb66"} Jul 07 07:12:02.134101 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:02.134084 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t" Jul 07 07:12:02.134602 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:02.134134 2579 scope.go:117] "RemoveContainer" containerID="3c56af0c7c8dcb092f4b4ad5483b72e4a5f4afc71ab25a3332baeaeb3b451096" Jul 07 07:12:02.161557 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:02.161532 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t"] Jul 07 07:12:02.165682 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:02.165660 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-94448f-7f9fb786cb-4f85t"] Jul 07 07:12:02.783422 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:02.783394 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58de2004-cf1b-45e1-9d36-cf1965ed5676" path="/var/lib/kubelet/pods/58de2004-cf1b-45e1-9d36-cf1965ed5676/volumes" Jul 07 07:12:04.889095 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:04.889069 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 07:12:04.894727 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:04.894708 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vn8hx_a328cd98-578d-4f2a-a622-0739620a9a6f/ovn-acl-logging/0.log" Jul 07 07:12:05.703734 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:05.703702 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:13.098787 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.098723 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" podUID="ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc" containerName="main" containerID="cri-o://f02cedbc6e9883026f349f5722390dd8636426c0c53ae44e2a7c5abf1112364a" gracePeriod=45 Jul 07 07:12:13.359623 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.359563 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:12:13.520281 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.520249 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-tls-certs\") pod \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " Jul 07 07:12:13.520489 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.520290 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-model-cache\") pod \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " Jul 07 07:12:13.520489 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.520334 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-dshm\") pod \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " Jul 07 07:12:13.520489 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.520435 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-tmp-dir\") pod \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " Jul 07 07:12:13.520489 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.520454 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-home\") pod \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " Jul 07 07:12:13.520720 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.520497 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45zbd\" (UniqueName: \"kubernetes.io/projected/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-kube-api-access-45zbd\") pod \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\" (UID: \"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc\") " Jul 07 07:12:13.520720 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.520610 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc" (UID: "ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:12:13.520830 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.520724 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-home" (OuterVolumeSpecName: "home") pod "ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc" (UID: "ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:12:13.520830 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.520741 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc" (UID: "ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:12:13.520905 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.520863 2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-tmp-dir\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:13.520905 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.520877 2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-home\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:13.520905 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.520886 2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-model-cache\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:13.522743 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.522715 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-kube-api-access-45zbd" (OuterVolumeSpecName: "kube-api-access-45zbd") pod "ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc" (UID: "ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc"). InnerVolumeSpecName "kube-api-access-45zbd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:12:13.522944 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.522918 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc" (UID: "ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:12:13.523025 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.522917 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-dshm" (OuterVolumeSpecName: "dshm") pod "ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc" (UID: "ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:12:13.622174 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.622069 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45zbd\" (UniqueName: \"kubernetes.io/projected/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-kube-api-access-45zbd\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:13.622174 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.622118 2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-tls-certs\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:13.622174 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:13.622128 2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc-dshm\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:14.199266 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:14.199229 2579 generic.go:356] "Generic (PLEG): container finished" podID="ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc" containerID="f02cedbc6e9883026f349f5722390dd8636426c0c53ae44e2a7c5abf1112364a" exitCode=0 Jul 07 07:12:14.199740 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:14.199272 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" Jul 07 07:12:14.199740 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:14.199307 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" event={"ID":"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc","Type":"ContainerDied","Data":"f02cedbc6e9883026f349f5722390dd8636426c0c53ae44e2a7c5abf1112364a"} Jul 07 07:12:14.199740 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:14.199345 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55" event={"ID":"ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc","Type":"ContainerDied","Data":"ecc006c0837a5e5f4fdd3b73dbab1c935dc8bafaf33132706740b62e2b5ce850"} Jul 07 07:12:14.199740 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:14.199387 2579 scope.go:117] "RemoveContainer" containerID="f02cedbc6e9883026f349f5722390dd8636426c0c53ae44e2a7c5abf1112364a" Jul 07 07:12:14.208858 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:14.208841 2579 scope.go:117] "RemoveContainer" containerID="f02cedbc6e9883026f349f5722390dd8636426c0c53ae44e2a7c5abf1112364a" Jul 07 07:12:14.209077 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:12:14.209062 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f02cedbc6e9883026f349f5722390dd8636426c0c53ae44e2a7c5abf1112364a\": container with ID starting with f02cedbc6e9883026f349f5722390dd8636426c0c53ae44e2a7c5abf1112364a not found: ID does not exist" containerID="f02cedbc6e9883026f349f5722390dd8636426c0c53ae44e2a7c5abf1112364a" Jul 07 07:12:14.209122 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:14.209085 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f02cedbc6e9883026f349f5722390dd8636426c0c53ae44e2a7c5abf1112364a"} err="failed to get container status \"f02cedbc6e9883026f349f5722390dd8636426c0c53ae44e2a7c5abf1112364a\": rpc error: code = NotFound desc = could not find container \"f02cedbc6e9883026f349f5722390dd8636426c0c53ae44e2a7c5abf1112364a\": container with ID starting with f02cedbc6e9883026f349f5722390dd8636426c0c53ae44e2a7c5abf1112364a not found: ID does not exist" Jul 07 07:12:14.221776 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:14.221752 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55"] Jul 07 07:12:14.227860 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:14.227840 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-94448f/rate-test-model-94448f-kserve-848f44dc67-9dc55"] Jul 07 07:12:14.783765 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:14.783733 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc" path="/var/lib/kubelet/pods/ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc/volumes" Jul 07 07:12:34.261725 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:34.261631 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:42.707166 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:42.707129 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:42.964433 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:42.964339 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2"] Jul 07 07:12:42.964626 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:42.964579 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" podUID="2008c626-76fe-41b5-9546-ba535db303f5" containerName="istio-proxy" containerID="cri-o://584db76973b7c9340f2e38e46c3e0691853ea116821604b8733fee3786361621" gracePeriod=30 Jul 07 07:12:42.991963 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:42.991935 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:43.226399 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:43.226308 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:43.411753 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:43.411722 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:43.843025 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:43.842993 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:43.942332 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:43.942298 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:44.044299 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.044258 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:44.100063 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.099989 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7"] Jul 07 07:12:44.100213 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.100191 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" podUID="97f050a1-5065-456f-a8d4-16dff3c9f2b4" containerName="maas-api" containerID="cri-o://93b4ebac0e43a586e96b4c8100b923e860eb936f8642a36cb1e8caa4a5ac5221" gracePeriod=30 Jul 07 07:12:44.136601 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.136564 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:44.348326 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.348299 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:12:44.358323 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.358249 2579 generic.go:356] "Generic (PLEG): container finished" podID="97f050a1-5065-456f-a8d4-16dff3c9f2b4" containerID="93b4ebac0e43a586e96b4c8100b923e860eb936f8642a36cb1e8caa4a5ac5221" exitCode=0 Jul 07 07:12:44.358323 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.358294 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" Jul 07 07:12:44.358545 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.358324 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" event={"ID":"97f050a1-5065-456f-a8d4-16dff3c9f2b4","Type":"ContainerDied","Data":"93b4ebac0e43a586e96b4c8100b923e860eb936f8642a36cb1e8caa4a5ac5221"} Jul 07 07:12:44.358545 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.358383 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7" event={"ID":"97f050a1-5065-456f-a8d4-16dff3c9f2b4","Type":"ContainerDied","Data":"caf27b1b645a07ae51f779dfe6c3701b6fe622196201cb18466663fa61ff6ea0"} Jul 07 07:12:44.358545 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.358406 2579 scope.go:117] "RemoveContainer" containerID="93b4ebac0e43a586e96b4c8100b923e860eb936f8642a36cb1e8caa4a5ac5221" Jul 07 07:12:44.366830 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.366812 2579 scope.go:117] "RemoveContainer" containerID="93b4ebac0e43a586e96b4c8100b923e860eb936f8642a36cb1e8caa4a5ac5221" Jul 07 07:12:44.367113 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:12:44.367090 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93b4ebac0e43a586e96b4c8100b923e860eb936f8642a36cb1e8caa4a5ac5221\": container with ID starting with 93b4ebac0e43a586e96b4c8100b923e860eb936f8642a36cb1e8caa4a5ac5221 not found: ID does not exist" containerID="93b4ebac0e43a586e96b4c8100b923e860eb936f8642a36cb1e8caa4a5ac5221" Jul 07 07:12:44.367156 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.367124 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93b4ebac0e43a586e96b4c8100b923e860eb936f8642a36cb1e8caa4a5ac5221"} err="failed to get container status \"93b4ebac0e43a586e96b4c8100b923e860eb936f8642a36cb1e8caa4a5ac5221\": rpc error: code = NotFound desc = could not find container \"93b4ebac0e43a586e96b4c8100b923e860eb936f8642a36cb1e8caa4a5ac5221\": container with ID starting with 93b4ebac0e43a586e96b4c8100b923e860eb936f8642a36cb1e8caa4a5ac5221 not found: ID does not exist" Jul 07 07:12:44.500593 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.500555 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwkvq\" (UniqueName: \"kubernetes.io/projected/97f050a1-5065-456f-a8d4-16dff3c9f2b4-kube-api-access-vwkvq\") pod \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\" (UID: \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\") " Jul 07 07:12:44.500762 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.500643 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97f050a1-5065-456f-a8d4-16dff3c9f2b4-maas-api-tls\") pod \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\" (UID: \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\") " Jul 07 07:12:44.500762 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.500689 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97f050a1-5065-456f-a8d4-16dff3c9f2b4-openshift-service-ca\") pod \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\" (UID: \"97f050a1-5065-456f-a8d4-16dff3c9f2b4\") " Jul 07 07:12:44.501084 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.501057 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97f050a1-5065-456f-a8d4-16dff3c9f2b4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "97f050a1-5065-456f-a8d4-16dff3c9f2b4" (UID: "97f050a1-5065-456f-a8d4-16dff3c9f2b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:12:44.502643 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.502617 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97f050a1-5065-456f-a8d4-16dff3c9f2b4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "97f050a1-5065-456f-a8d4-16dff3c9f2b4" (UID: "97f050a1-5065-456f-a8d4-16dff3c9f2b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 07:12:44.502756 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.502643 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97f050a1-5065-456f-a8d4-16dff3c9f2b4-kube-api-access-vwkvq" (OuterVolumeSpecName: "kube-api-access-vwkvq") pod "97f050a1-5065-456f-a8d4-16dff3c9f2b4" (UID: "97f050a1-5065-456f-a8d4-16dff3c9f2b4"). InnerVolumeSpecName "kube-api-access-vwkvq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:12:44.601703 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.601669 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97f050a1-5065-456f-a8d4-16dff3c9f2b4-openshift-service-ca\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:44.601703 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.601698 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwkvq\" (UniqueName: \"kubernetes.io/projected/97f050a1-5065-456f-a8d4-16dff3c9f2b4-kube-api-access-vwkvq\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:44.601703 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.601708 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97f050a1-5065-456f-a8d4-16dff3c9f2b4-maas-api-tls\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:44.698694 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.698661 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7"] Jul 07 07:12:44.702802 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.702776 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c72c9a-5946b84c99-ftjt7"] Jul 07 07:12:44.783304 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:44.783273 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97f050a1-5065-456f-a8d4-16dff3c9f2b4" path="/var/lib/kubelet/pods/97f050a1-5065-456f-a8d4-16dff3c9f2b4/volumes" Jul 07 07:12:45.259741 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:45.259703 2579 patch_prober.go:28] interesting pod/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 07:12:45.260097 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:45.259760 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" podUID="2008c626-76fe-41b5-9546-ba535db303f5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 07:12:48.225525 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.225464 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:12:48.338107 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.338080 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-workload-socket\") pod \"2008c626-76fe-41b5-9546-ba535db303f5\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " Jul 07 07:12:48.338107 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.338109 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2008c626-76fe-41b5-9546-ba535db303f5-istio-podinfo\") pod \"2008c626-76fe-41b5-9546-ba535db303f5\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " Jul 07 07:12:48.338359 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.338188 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2008c626-76fe-41b5-9546-ba535db303f5-istiod-ca-cert\") pod \"2008c626-76fe-41b5-9546-ba535db303f5\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " Jul 07 07:12:48.338359 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.338209 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2008c626-76fe-41b5-9546-ba535db303f5-istio-token\") pod \"2008c626-76fe-41b5-9546-ba535db303f5\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " Jul 07 07:12:48.338359 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.338232 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-workload-certs\") pod \"2008c626-76fe-41b5-9546-ba535db303f5\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " Jul 07 07:12:48.338359 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.338275 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-istio-envoy\") pod \"2008c626-76fe-41b5-9546-ba535db303f5\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " Jul 07 07:12:48.338359 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.338304 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xstt9\" (UniqueName: \"kubernetes.io/projected/2008c626-76fe-41b5-9546-ba535db303f5-kube-api-access-xstt9\") pod \"2008c626-76fe-41b5-9546-ba535db303f5\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " Jul 07 07:12:48.338680 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.338409 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2008c626-76fe-41b5-9546-ba535db303f5" (UID: "2008c626-76fe-41b5-9546-ba535db303f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:12:48.338680 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.338659 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2008c626-76fe-41b5-9546-ba535db303f5" (UID: "2008c626-76fe-41b5-9546-ba535db303f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:12:48.338795 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.338718 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-istio-data\") pod \"2008c626-76fe-41b5-9546-ba535db303f5\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " Jul 07 07:12:48.338795 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.338727 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2008c626-76fe-41b5-9546-ba535db303f5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2008c626-76fe-41b5-9546-ba535db303f5" (UID: "2008c626-76fe-41b5-9546-ba535db303f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:12:48.338795 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.338782 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-credential-socket\") pod \"2008c626-76fe-41b5-9546-ba535db303f5\" (UID: \"2008c626-76fe-41b5-9546-ba535db303f5\") " Jul 07 07:12:48.339072 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.339047 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2008c626-76fe-41b5-9546-ba535db303f5" (UID: "2008c626-76fe-41b5-9546-ba535db303f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:12:48.339150 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.339134 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-workload-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:48.339206 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.339157 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2008c626-76fe-41b5-9546-ba535db303f5-istiod-ca-cert\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:48.339206 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.339171 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-workload-certs\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:48.339206 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.339183 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-credential-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:48.339334 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.339252 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-istio-data" (OuterVolumeSpecName: "istio-data") pod "2008c626-76fe-41b5-9546-ba535db303f5" (UID: "2008c626-76fe-41b5-9546-ba535db303f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:12:48.340556 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.340531 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2008c626-76fe-41b5-9546-ba535db303f5-kube-api-access-xstt9" (OuterVolumeSpecName: "kube-api-access-xstt9") pod "2008c626-76fe-41b5-9546-ba535db303f5" (UID: "2008c626-76fe-41b5-9546-ba535db303f5"). InnerVolumeSpecName "kube-api-access-xstt9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:12:48.340685 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.340607 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2008c626-76fe-41b5-9546-ba535db303f5-istio-token" (OuterVolumeSpecName: "istio-token") pod "2008c626-76fe-41b5-9546-ba535db303f5" (UID: "2008c626-76fe-41b5-9546-ba535db303f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:12:48.340808 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.340787 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2008c626-76fe-41b5-9546-ba535db303f5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2008c626-76fe-41b5-9546-ba535db303f5" (UID: "2008c626-76fe-41b5-9546-ba535db303f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 07:12:48.340873 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.340794 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2008c626-76fe-41b5-9546-ba535db303f5" (UID: "2008c626-76fe-41b5-9546-ba535db303f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:12:48.387060 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.387027 2579 generic.go:356] "Generic (PLEG): container finished" podID="2008c626-76fe-41b5-9546-ba535db303f5" containerID="584db76973b7c9340f2e38e46c3e0691853ea116821604b8733fee3786361621" exitCode=0 Jul 07 07:12:48.387174 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.387089 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" Jul 07 07:12:48.387174 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.387108 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" event={"ID":"2008c626-76fe-41b5-9546-ba535db303f5","Type":"ContainerDied","Data":"584db76973b7c9340f2e38e46c3e0691853ea116821604b8733fee3786361621"} Jul 07 07:12:48.387289 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.387180 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2" event={"ID":"2008c626-76fe-41b5-9546-ba535db303f5","Type":"ContainerDied","Data":"494c78b0830b9f06f9b63973f9ecaa8af52907741df396c4da0787b1ef2eeca1"} Jul 07 07:12:48.387289 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.387201 2579 scope.go:117] "RemoveContainer" containerID="584db76973b7c9340f2e38e46c3e0691853ea116821604b8733fee3786361621" Jul 07 07:12:48.396465 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.396449 2579 scope.go:117] "RemoveContainer" containerID="d8cc6b1751c6c316e7362f66b279f3f09dfefae802473879cb204054d8072dee" Jul 07 07:12:48.404271 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.404254 2579 scope.go:117] "RemoveContainer" containerID="584db76973b7c9340f2e38e46c3e0691853ea116821604b8733fee3786361621" Jul 07 07:12:48.404518 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:12:48.404501 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"584db76973b7c9340f2e38e46c3e0691853ea116821604b8733fee3786361621\": container with ID starting with 584db76973b7c9340f2e38e46c3e0691853ea116821604b8733fee3786361621 not found: ID does not exist" containerID="584db76973b7c9340f2e38e46c3e0691853ea116821604b8733fee3786361621" Jul 07 07:12:48.404576 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.404525 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"584db76973b7c9340f2e38e46c3e0691853ea116821604b8733fee3786361621"} err="failed to get container status \"584db76973b7c9340f2e38e46c3e0691853ea116821604b8733fee3786361621\": rpc error: code = NotFound desc = could not find container \"584db76973b7c9340f2e38e46c3e0691853ea116821604b8733fee3786361621\": container with ID starting with 584db76973b7c9340f2e38e46c3e0691853ea116821604b8733fee3786361621 not found: ID does not exist" Jul 07 07:12:48.404576 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.404541 2579 scope.go:117] "RemoveContainer" containerID="d8cc6b1751c6c316e7362f66b279f3f09dfefae802473879cb204054d8072dee" Jul 07 07:12:48.404742 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:12:48.404728 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8cc6b1751c6c316e7362f66b279f3f09dfefae802473879cb204054d8072dee\": container with ID starting with d8cc6b1751c6c316e7362f66b279f3f09dfefae802473879cb204054d8072dee not found: ID does not exist" containerID="d8cc6b1751c6c316e7362f66b279f3f09dfefae802473879cb204054d8072dee" Jul 07 07:12:48.404781 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.404746 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8cc6b1751c6c316e7362f66b279f3f09dfefae802473879cb204054d8072dee"} err="failed to get container status \"d8cc6b1751c6c316e7362f66b279f3f09dfefae802473879cb204054d8072dee\": rpc error: code = NotFound desc = could not find container \"d8cc6b1751c6c316e7362f66b279f3f09dfefae802473879cb204054d8072dee\": container with ID starting with d8cc6b1751c6c316e7362f66b279f3f09dfefae802473879cb204054d8072dee not found: ID does not exist" Jul 07 07:12:48.416644 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.416624 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2"] Jul 07 07:12:48.424921 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.424883 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-94448f-openshift-default-97fcf6447-gxnf2"] Jul 07 07:12:48.439938 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.439916 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2008c626-76fe-41b5-9546-ba535db303f5-istio-podinfo\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:48.440046 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.439941 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2008c626-76fe-41b5-9546-ba535db303f5-istio-token\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:48.440046 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.439956 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-istio-envoy\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:48.440046 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.439969 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xstt9\" (UniqueName: \"kubernetes.io/projected/2008c626-76fe-41b5-9546-ba535db303f5-kube-api-access-xstt9\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:48.440046 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.439981 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2008c626-76fe-41b5-9546-ba535db303f5-istio-data\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:48.783652 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:48.783625 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2008c626-76fe-41b5-9546-ba535db303f5" path="/var/lib/kubelet/pods/2008c626-76fe-41b5-9546-ba535db303f5/volumes" Jul 07 07:12:50.242155 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:50.242114 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:50.357946 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:50.357916 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:50.523909 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:50.523830 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59"] Jul 07 07:12:50.524059 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:50.524008 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" podUID="a94128ba-f6be-4046-8bc6-8bf726881ef0" containerName="istio-proxy" containerID="cri-o://eedb9ed673ecdc7868436cfb3ecf0b42b566d0be515c679bf7ba73a315b04d41" gracePeriod=30 Jul 07 07:12:50.542671 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:50.542642 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:50.755424 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:50.755387 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:12:55.780199 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.780174 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:12:55.911340 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.911248 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-workload-certs\") pod \"a94128ba-f6be-4046-8bc6-8bf726881ef0\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " Jul 07 07:12:55.911340 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.911286 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-workload-socket\") pod \"a94128ba-f6be-4046-8bc6-8bf726881ef0\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " Jul 07 07:12:55.911340 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.911321 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a94128ba-f6be-4046-8bc6-8bf726881ef0-istiod-ca-cert\") pod \"a94128ba-f6be-4046-8bc6-8bf726881ef0\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " Jul 07 07:12:55.911652 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.911349 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9z5ns\" (UniqueName: \"kubernetes.io/projected/a94128ba-f6be-4046-8bc6-8bf726881ef0-kube-api-access-9z5ns\") pod \"a94128ba-f6be-4046-8bc6-8bf726881ef0\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " Jul 07 07:12:55.911652 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.911447 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-podinfo\") pod \"a94128ba-f6be-4046-8bc6-8bf726881ef0\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " Jul 07 07:12:55.911652 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.911478 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-data\") pod \"a94128ba-f6be-4046-8bc6-8bf726881ef0\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " Jul 07 07:12:55.911652 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.911502 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-envoy\") pod \"a94128ba-f6be-4046-8bc6-8bf726881ef0\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " Jul 07 07:12:55.911652 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.911519 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a94128ba-f6be-4046-8bc6-8bf726881ef0" (UID: "a94128ba-f6be-4046-8bc6-8bf726881ef0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:12:55.911652 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.911607 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a94128ba-f6be-4046-8bc6-8bf726881ef0" (UID: "a94128ba-f6be-4046-8bc6-8bf726881ef0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:12:55.911652 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.911647 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-credential-socket\") pod \"a94128ba-f6be-4046-8bc6-8bf726881ef0\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " Jul 07 07:12:55.912018 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.911683 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-token\") pod \"a94128ba-f6be-4046-8bc6-8bf726881ef0\" (UID: \"a94128ba-f6be-4046-8bc6-8bf726881ef0\") " Jul 07 07:12:55.912018 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.911892 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a94128ba-f6be-4046-8bc6-8bf726881ef0" (UID: "a94128ba-f6be-4046-8bc6-8bf726881ef0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:12:55.912018 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.911914 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a94128ba-f6be-4046-8bc6-8bf726881ef0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a94128ba-f6be-4046-8bc6-8bf726881ef0" (UID: "a94128ba-f6be-4046-8bc6-8bf726881ef0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 07:12:55.912171 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.912049 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-credential-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:55.912171 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.912071 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-workload-certs\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:55.912171 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.912085 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-workload-socket\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:55.912171 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.912098 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a94128ba-f6be-4046-8bc6-8bf726881ef0-istiod-ca-cert\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:55.912410 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.912352 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-data" (OuterVolumeSpecName: "istio-data") pod "a94128ba-f6be-4046-8bc6-8bf726881ef0" (UID: "a94128ba-f6be-4046-8bc6-8bf726881ef0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:12:55.913811 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.913783 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-token" (OuterVolumeSpecName: "istio-token") pod "a94128ba-f6be-4046-8bc6-8bf726881ef0" (UID: "a94128ba-f6be-4046-8bc6-8bf726881ef0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:12:55.913811 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.913791 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a94128ba-f6be-4046-8bc6-8bf726881ef0-kube-api-access-9z5ns" (OuterVolumeSpecName: "kube-api-access-9z5ns") pod "a94128ba-f6be-4046-8bc6-8bf726881ef0" (UID: "a94128ba-f6be-4046-8bc6-8bf726881ef0"). InnerVolumeSpecName "kube-api-access-9z5ns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 07:12:55.914105 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.914083 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a94128ba-f6be-4046-8bc6-8bf726881ef0" (UID: "a94128ba-f6be-4046-8bc6-8bf726881ef0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 07:12:55.914145 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:55.914121 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a94128ba-f6be-4046-8bc6-8bf726881ef0" (UID: "a94128ba-f6be-4046-8bc6-8bf726881ef0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 07:12:56.013350 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.013317 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-podinfo\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:56.013350 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.013344 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-data\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:56.013350 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.013355 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-envoy\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:56.013579 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.013377 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a94128ba-f6be-4046-8bc6-8bf726881ef0-istio-token\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:56.013579 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.013390 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z5ns\" (UniqueName: \"kubernetes.io/projected/a94128ba-f6be-4046-8bc6-8bf726881ef0-kube-api-access-9z5ns\") on node \"ip-10-0-130-176.ec2.internal\" DevicePath \"\"" Jul 07 07:12:56.166614 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.166538 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-858ccdc565-w6cgq_75b63629-9690-44c7-8263-74cba91fbbfb/maas-api/0.log" Jul 07 07:12:56.288827 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.288795 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77b55b5c58-wt88z_7a16f5b6-099a-4204-8c82-d28b8a48732c/manager/0.log" Jul 07 07:12:56.430250 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.430163 2579 generic.go:356] "Generic (PLEG): container finished" podID="a94128ba-f6be-4046-8bc6-8bf726881ef0" containerID="eedb9ed673ecdc7868436cfb3ecf0b42b566d0be515c679bf7ba73a315b04d41" exitCode=0 Jul 07 07:12:56.430250 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.430208 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" Jul 07 07:12:56.430506 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.430242 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" event={"ID":"a94128ba-f6be-4046-8bc6-8bf726881ef0","Type":"ContainerDied","Data":"eedb9ed673ecdc7868436cfb3ecf0b42b566d0be515c679bf7ba73a315b04d41"} Jul 07 07:12:56.430506 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.430299 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59" event={"ID":"a94128ba-f6be-4046-8bc6-8bf726881ef0","Type":"ContainerDied","Data":"f4519ebd8774ce7641bf41bcce8aa10175f6dca91f6d043335a0ab0a0f97f585"} Jul 07 07:12:56.430506 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.430331 2579 scope.go:117] "RemoveContainer" containerID="eedb9ed673ecdc7868436cfb3ecf0b42b566d0be515c679bf7ba73a315b04d41" Jul 07 07:12:56.440453 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.440438 2579 scope.go:117] "RemoveContainer" containerID="3fe6748c62578baa1f77e0b46d14cdea24c2ef8f1b28fd5575b435c594096533" Jul 07 07:12:56.448534 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.448521 2579 scope.go:117] "RemoveContainer" containerID="eedb9ed673ecdc7868436cfb3ecf0b42b566d0be515c679bf7ba73a315b04d41" Jul 07 07:12:56.448815 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:12:56.448796 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eedb9ed673ecdc7868436cfb3ecf0b42b566d0be515c679bf7ba73a315b04d41\": container with ID starting with eedb9ed673ecdc7868436cfb3ecf0b42b566d0be515c679bf7ba73a315b04d41 not found: ID does not exist" containerID="eedb9ed673ecdc7868436cfb3ecf0b42b566d0be515c679bf7ba73a315b04d41" Jul 07 07:12:56.448871 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.448822 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eedb9ed673ecdc7868436cfb3ecf0b42b566d0be515c679bf7ba73a315b04d41"} err="failed to get container status \"eedb9ed673ecdc7868436cfb3ecf0b42b566d0be515c679bf7ba73a315b04d41\": rpc error: code = NotFound desc = could not find container \"eedb9ed673ecdc7868436cfb3ecf0b42b566d0be515c679bf7ba73a315b04d41\": container with ID starting with eedb9ed673ecdc7868436cfb3ecf0b42b566d0be515c679bf7ba73a315b04d41 not found: ID does not exist" Jul 07 07:12:56.448871 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.448840 2579 scope.go:117] "RemoveContainer" containerID="3fe6748c62578baa1f77e0b46d14cdea24c2ef8f1b28fd5575b435c594096533" Jul 07 07:12:56.449060 ip-10-0-130-176 kubenswrapper[2579]: E0707 07:12:56.449042 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fe6748c62578baa1f77e0b46d14cdea24c2ef8f1b28fd5575b435c594096533\": container with ID starting with 3fe6748c62578baa1f77e0b46d14cdea24c2ef8f1b28fd5575b435c594096533 not found: ID does not exist" containerID="3fe6748c62578baa1f77e0b46d14cdea24c2ef8f1b28fd5575b435c594096533" Jul 07 07:12:56.449110 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.449069 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fe6748c62578baa1f77e0b46d14cdea24c2ef8f1b28fd5575b435c594096533"} err="failed to get container status \"3fe6748c62578baa1f77e0b46d14cdea24c2ef8f1b28fd5575b435c594096533\": rpc error: code = NotFound desc = could not find container \"3fe6748c62578baa1f77e0b46d14cdea24c2ef8f1b28fd5575b435c594096533\": container with ID starting with 3fe6748c62578baa1f77e0b46d14cdea24c2ef8f1b28fd5575b435c594096533 not found: ID does not exist" Jul 07 07:12:56.454019 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.453998 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59"] Jul 07 07:12:56.460911 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.460891 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-c72c9a-openshift-default-64fcf54f9f-8bh59"] Jul 07 07:12:56.783248 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.783207 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a94128ba-f6be-4046-8bc6-8bf726881ef0" path="/var/lib/kubelet/pods/a94128ba-f6be-4046-8bc6-8bf726881ef0/volumes" Jul 07 07:12:56.891175 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.891153 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c66c8b568-ljkmw_1cd7caa7-5d44-4786-bda1-ef41b106db29/copy-manifests/0.log" Jul 07 07:12:56.898494 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:56.898476 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c66c8b568-ljkmw_1cd7caa7-5d44-4786-bda1-ef41b106db29/manager/0.log" Jul 07 07:12:57.775002 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:57.774973 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx_76847b50-0e20-4c4d-a394-7144cc003b8a/util/0.log" Jul 07 07:12:57.781792 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:57.781766 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx_76847b50-0e20-4c4d-a394-7144cc003b8a/pull/0.log" Jul 07 07:12:57.788429 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:57.788391 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx_76847b50-0e20-4c4d-a394-7144cc003b8a/extract/0.log" Jul 07 07:12:57.902031 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:57.902003 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5_530b35ad-628c-4f7a-b838-fd84a0924eec/util/0.log" Jul 07 07:12:57.908810 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:57.908780 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5_530b35ad-628c-4f7a-b838-fd84a0924eec/pull/0.log" Jul 07 07:12:57.917895 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:57.917879 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5_530b35ad-628c-4f7a-b838-fd84a0924eec/extract/0.log" Jul 07 07:12:58.025939 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:58.025918 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8_2c289d7b-c836-49e8-877e-05284e3f4d36/util/0.log" Jul 07 07:12:58.031322 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:58.031302 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8_2c289d7b-c836-49e8-877e-05284e3f4d36/pull/0.log" Jul 07 07:12:58.037640 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:58.037621 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8_2c289d7b-c836-49e8-877e-05284e3f4d36/extract/0.log" Jul 07 07:12:58.156785 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:58.156756 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44_380a39e7-ed43-402a-b786-99b16101c5d1/util/0.log" Jul 07 07:12:58.163312 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:58.163270 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44_380a39e7-ed43-402a-b786-99b16101c5d1/pull/0.log" Jul 07 07:12:58.169335 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:58.169318 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44_380a39e7-ed43-402a-b786-99b16101c5d1/extract/0.log" Jul 07 07:12:59.481568 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:59.481520 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf42bvt5_1057d10c-fc71-480b-8f6a-1a6f4832748c/istio-proxy/0.log" Jul 07 07:12:59.601831 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:59.601791 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2gjqq_230065e6-ce67-4fbc-95bf-dff89318b04a/discovery/0.log" Jul 07 07:12:59.949241 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:12:59.949210 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rrfhg_b917a0bf-164b-40e9-88a6-ad5ac62e29ca/istio-proxy/1.log" Jul 07 07:13:00.648864 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:00.648811 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-fsjvc_67945122-a2c9-49e3-a6b1-979ce437c9fe/main/0.log" Jul 07 07:13:05.336169 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.336129 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:13:05.337942 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.337911 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f8h84/must-gather-2c69g"] Jul 07 07:13:05.338700 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.338675 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94128ba-f6be-4046-8bc6-8bf726881ef0" containerName="istio-proxy" Jul 07 07:13:05.338700 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.338700 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94128ba-f6be-4046-8bc6-8bf726881ef0" containerName="istio-proxy" Jul 07 07:13:05.338865 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.338729 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2008c626-76fe-41b5-9546-ba535db303f5" containerName="istio-proxy" Jul 07 07:13:05.338865 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.338738 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2008c626-76fe-41b5-9546-ba535db303f5" containerName="istio-proxy" Jul 07 07:13:05.338865 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.338757 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc" containerName="main" Jul 07 07:13:05.338865 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.338767 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc" containerName="main" Jul 07 07:13:05.338865 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.338785 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2008c626-76fe-41b5-9546-ba535db303f5" containerName="istio-proxy" Jul 07 07:13:05.338865 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.338798 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2008c626-76fe-41b5-9546-ba535db303f5" containerName="istio-proxy" Jul 07 07:13:05.338865 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.338830 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97f050a1-5065-456f-a8d4-16dff3c9f2b4" containerName="maas-api" Jul 07 07:13:05.338865 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.338839 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="97f050a1-5065-456f-a8d4-16dff3c9f2b4" containerName="maas-api" Jul 07 07:13:05.338865 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.338853 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58de2004-cf1b-45e1-9d36-cf1965ed5676" containerName="maas-api" Jul 07 07:13:05.338865 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.338861 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="58de2004-cf1b-45e1-9d36-cf1965ed5676" containerName="maas-api" Jul 07 07:13:05.339323 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.339005 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="97f050a1-5065-456f-a8d4-16dff3c9f2b4" containerName="maas-api" Jul 07 07:13:05.339323 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.339017 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="58de2004-cf1b-45e1-9d36-cf1965ed5676" containerName="maas-api" Jul 07 07:13:05.339323 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.339026 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2008c626-76fe-41b5-9546-ba535db303f5" containerName="istio-proxy" Jul 07 07:13:05.339323 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.339040 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba0c6ed3-5124-4a6b-97a3-ce157f1da5dc" containerName="main" Jul 07 07:13:05.339323 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.339048 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94128ba-f6be-4046-8bc6-8bf726881ef0" containerName="istio-proxy" Jul 07 07:13:05.339323 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.339061 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2008c626-76fe-41b5-9546-ba535db303f5" containerName="istio-proxy" Jul 07 07:13:05.339323 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.339203 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94128ba-f6be-4046-8bc6-8bf726881ef0" containerName="istio-proxy" Jul 07 07:13:05.339323 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.339216 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94128ba-f6be-4046-8bc6-8bf726881ef0" containerName="istio-proxy" Jul 07 07:13:05.339703 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.339430 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94128ba-f6be-4046-8bc6-8bf726881ef0" containerName="istio-proxy" Jul 07 07:13:05.345098 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.345079 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f8h84/must-gather-2c69g" Jul 07 07:13:05.348138 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.348119 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f8h84\"/\"openshift-service-ca.crt\"" Jul 07 07:13:05.349447 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.349427 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f8h84\"/\"kube-root-ca.crt\"" Jul 07 07:13:05.349549 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.349474 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-f8h84\"/\"default-dockercfg-xq6kg\"" Jul 07 07:13:05.352035 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.352013 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f8h84/must-gather-2c69g"] Jul 07 07:13:05.386564 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.386534 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27ws9\" (UniqueName: \"kubernetes.io/projected/a4dbfef3-1d0a-4dbd-b58d-9b1a26ea1c96-kube-api-access-27ws9\") pod \"must-gather-2c69g\" (UID: \"a4dbfef3-1d0a-4dbd-b58d-9b1a26ea1c96\") " pod="openshift-must-gather-f8h84/must-gather-2c69g" Jul 07 07:13:05.386733 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.386610 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a4dbfef3-1d0a-4dbd-b58d-9b1a26ea1c96-must-gather-output\") pod \"must-gather-2c69g\" (UID: \"a4dbfef3-1d0a-4dbd-b58d-9b1a26ea1c96\") " pod="openshift-must-gather-f8h84/must-gather-2c69g" Jul 07 07:13:05.460988 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.460959 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" limit=1024 Jul 07 07:13:05.487416 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.487384 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a4dbfef3-1d0a-4dbd-b58d-9b1a26ea1c96-must-gather-output\") pod \"must-gather-2c69g\" (UID: \"a4dbfef3-1d0a-4dbd-b58d-9b1a26ea1c96\") " pod="openshift-must-gather-f8h84/must-gather-2c69g" Jul 07 07:13:05.487535 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.487453 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27ws9\" (UniqueName: \"kubernetes.io/projected/a4dbfef3-1d0a-4dbd-b58d-9b1a26ea1c96-kube-api-access-27ws9\") pod \"must-gather-2c69g\" (UID: \"a4dbfef3-1d0a-4dbd-b58d-9b1a26ea1c96\") " pod="openshift-must-gather-f8h84/must-gather-2c69g" Jul 07 07:13:05.487667 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.487650 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a4dbfef3-1d0a-4dbd-b58d-9b1a26ea1c96-must-gather-output\") pod \"must-gather-2c69g\" (UID: \"a4dbfef3-1d0a-4dbd-b58d-9b1a26ea1c96\") " pod="openshift-must-gather-f8h84/must-gather-2c69g" Jul 07 07:13:05.500445 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.500417 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27ws9\" (UniqueName: \"kubernetes.io/projected/a4dbfef3-1d0a-4dbd-b58d-9b1a26ea1c96-kube-api-access-27ws9\") pod \"must-gather-2c69g\" (UID: \"a4dbfef3-1d0a-4dbd-b58d-9b1a26ea1c96\") " pod="openshift-must-gather-f8h84/must-gather-2c69g" Jul 07 07:13:05.655450 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.655351 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f8h84/must-gather-2c69g" Jul 07 07:13:05.790478 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.790455 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f8h84/must-gather-2c69g"] Jul 07 07:13:05.792803 ip-10-0-130-176 kubenswrapper[2579]: W0707 07:13:05.792776 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4dbfef3_1d0a_4dbd_b58d_9b1a26ea1c96.slice/crio-fd5a794d7d2ba1c7b105a5c2ff686ec60e4ca5bad8d695daa461a7ee1cc6689c WatchSource:0}: Error finding container fd5a794d7d2ba1c7b105a5c2ff686ec60e4ca5bad8d695daa461a7ee1cc6689c: Status 404 returned error can't find the container with id fd5a794d7d2ba1c7b105a5c2ff686ec60e4ca5bad8d695daa461a7ee1cc6689c Jul 07 07:13:05.794636 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:05.794617 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 07:13:06.485507 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:06.485478 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f8h84/must-gather-2c69g" event={"ID":"a4dbfef3-1d0a-4dbd-b58d-9b1a26ea1c96","Type":"ContainerStarted","Data":"fd5a794d7d2ba1c7b105a5c2ff686ec60e4ca5bad8d695daa461a7ee1cc6689c"} Jul 07 07:13:07.496406 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:07.496336 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f8h84/must-gather-2c69g" event={"ID":"a4dbfef3-1d0a-4dbd-b58d-9b1a26ea1c96","Type":"ContainerStarted","Data":"4cb91fb9693929964e848e71557c4797ba89e294c4ab0e2af1b753e0723088b3"} Jul 07 07:13:07.496875 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:07.496426 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f8h84/must-gather-2c69g" event={"ID":"a4dbfef3-1d0a-4dbd-b58d-9b1a26ea1c96","Type":"ContainerStarted","Data":"8aef934161336fdf8483d850150d2bb42b6ab1cb42ddd08dc4a743b396540db2"} Jul 07 07:13:08.368159 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:08.368125 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4bdk4_997fb304-b3ae-42e0-8a84-db42daea16b1/global-pull-secret-syncer/0.log" Jul 07 07:13:08.501135 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:08.501098 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5q8w9_606bdf3e-ea47-4a60-a2fd-cf5aa63d341c/konnectivity-agent/0.log" Jul 07 07:13:08.605913 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:08.605884 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-176.ec2.internal_b737b79e957a55e359ff156469d79a5e/haproxy/0.log" Jul 07 07:13:12.498018 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:12.497990 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx_76847b50-0e20-4c4d-a394-7144cc003b8a/extract/0.log" Jul 07 07:13:12.518381 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:12.518342 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx_76847b50-0e20-4c4d-a394-7144cc003b8a/util/0.log" Jul 07 07:13:12.541389 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:12.541347 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cblzx_76847b50-0e20-4c4d-a394-7144cc003b8a/pull/0.log" Jul 07 07:13:12.569380 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:12.569319 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5_530b35ad-628c-4f7a-b838-fd84a0924eec/extract/0.log" Jul 07 07:13:12.588692 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:12.588655 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5_530b35ad-628c-4f7a-b838-fd84a0924eec/util/0.log" Jul 07 07:13:12.608097 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:12.608045 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zhnh5_530b35ad-628c-4f7a-b838-fd84a0924eec/pull/0.log" Jul 07 07:13:12.640670 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:12.640645 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8_2c289d7b-c836-49e8-877e-05284e3f4d36/extract/0.log" Jul 07 07:13:12.663843 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:12.663816 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8_2c289d7b-c836-49e8-877e-05284e3f4d36/util/0.log" Jul 07 07:13:12.688834 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:12.688803 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736m5b8_2c289d7b-c836-49e8-877e-05284e3f4d36/pull/0.log" Jul 07 07:13:12.712918 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:12.712882 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44_380a39e7-ed43-402a-b786-99b16101c5d1/extract/0.log" Jul 07 07:13:12.733042 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:12.733013 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44_380a39e7-ed43-402a-b786-99b16101c5d1/util/0.log" Jul 07 07:13:12.757844 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:12.757726 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdm44_380a39e7-ed43-402a-b786-99b16101c5d1/pull/0.log" Jul 07 07:13:14.835269 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:14.835230 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f740c654-b39f-4226-acf0-bcd17a3f7b40/alertmanager/0.log" Jul 07 07:13:14.862855 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:14.862823 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f740c654-b39f-4226-acf0-bcd17a3f7b40/config-reloader/0.log" Jul 07 07:13:14.886178 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:14.886140 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f740c654-b39f-4226-acf0-bcd17a3f7b40/kube-rbac-proxy-web/0.log" Jul 07 07:13:14.905576 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:14.905488 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f740c654-b39f-4226-acf0-bcd17a3f7b40/kube-rbac-proxy/0.log" Jul 07 07:13:14.926498 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:14.926469 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f740c654-b39f-4226-acf0-bcd17a3f7b40/kube-rbac-proxy-metric/0.log" Jul 07 07:13:14.948118 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:14.948084 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f740c654-b39f-4226-acf0-bcd17a3f7b40/prom-label-proxy/0.log" Jul 07 07:13:14.988101 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:14.988054 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f740c654-b39f-4226-acf0-bcd17a3f7b40/init-config-reloader/0.log" Jul 07 07:13:15.086172 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:15.086089 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-g7lvj_5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe/kube-state-metrics/0.log" Jul 07 07:13:15.151876 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:15.151848 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-g7lvj_5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe/kube-rbac-proxy-main/0.log" Jul 07 07:13:15.208961 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:15.208936 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-g7lvj_5e3ca3ab-5ff9-4f9d-b5b2-646ee1219efe/kube-rbac-proxy-self/0.log" Jul 07 07:13:15.402950 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:15.402852 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cfn7q_f6837d75-a0c8-4dee-a2b1-f5844de3f0e8/node-exporter/0.log" Jul 07 07:13:15.434906 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:15.434867 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cfn7q_f6837d75-a0c8-4dee-a2b1-f5844de3f0e8/kube-rbac-proxy/0.log" Jul 07 07:13:15.468301 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:15.468271 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cfn7q_f6837d75-a0c8-4dee-a2b1-f5844de3f0e8/init-textfile/0.log" Jul 07 07:13:15.965981 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:15.965954 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8fbcf4c6d-xtd82_c8e9b298-5749-45ad-afd4-b7de45d3a2e6/telemeter-client/0.log" Jul 07 07:13:15.990978 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:15.990914 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8fbcf4c6d-xtd82_c8e9b298-5749-45ad-afd4-b7de45d3a2e6/reload/0.log" Jul 07 07:13:16.010959 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:16.010921 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8fbcf4c6d-xtd82_c8e9b298-5749-45ad-afd4-b7de45d3a2e6/kube-rbac-proxy/0.log" Jul 07 07:13:16.047779 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:16.047679 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f6bb9757c-s52tl_5c3bb6d4-279f-4c40-af27-053f8f50b538/thanos-query/0.log" Jul 07 07:13:16.081542 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:16.081507 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f6bb9757c-s52tl_5c3bb6d4-279f-4c40-af27-053f8f50b538/kube-rbac-proxy-web/0.log" Jul 07 07:13:16.104802 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:16.104750 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f6bb9757c-s52tl_5c3bb6d4-279f-4c40-af27-053f8f50b538/kube-rbac-proxy/0.log" Jul 07 07:13:16.129865 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:16.129829 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f6bb9757c-s52tl_5c3bb6d4-279f-4c40-af27-053f8f50b538/prom-label-proxy/0.log" Jul 07 07:13:16.152476 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:16.152442 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f6bb9757c-s52tl_5c3bb6d4-279f-4c40-af27-053f8f50b538/kube-rbac-proxy-rules/0.log" Jul 07 07:13:16.172413 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:16.172348 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f6bb9757c-s52tl_5c3bb6d4-279f-4c40-af27-053f8f50b538/kube-rbac-proxy-metrics/0.log" Jul 07 07:13:18.788224 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:18.788162 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f8h84/must-gather-2c69g" podStartSLOduration=12.762185872 podStartE2EDuration="13.788140403s" totalImagesPullingTime="1.025954531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 07:13:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:13:07.515789543 +0000 UTC m=+2463.355630511" watchObservedRunningTime="2026-07-07 07:13:18.788140403 +0000 UTC m=+2474.627981371" Jul 07 07:13:18.788865 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:18.788690 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv"] Jul 07 07:13:18.796884 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:18.796853 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:18.809558 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:18.809530 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv"] Jul 07 07:13:18.932280 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:18.932231 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25442d04-e49f-454d-a510-296bc88e3deb-lib-modules\") pod \"perf-node-gather-daemonset-tbjtv\" (UID: \"25442d04-e49f-454d-a510-296bc88e3deb\") " pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:18.932488 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:18.932301 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/25442d04-e49f-454d-a510-296bc88e3deb-podres\") pod \"perf-node-gather-daemonset-tbjtv\" (UID: \"25442d04-e49f-454d-a510-296bc88e3deb\") " pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:18.932488 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:18.932411 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbhvm\" (UniqueName: \"kubernetes.io/projected/25442d04-e49f-454d-a510-296bc88e3deb-kube-api-access-wbhvm\") pod \"perf-node-gather-daemonset-tbjtv\" (UID: \"25442d04-e49f-454d-a510-296bc88e3deb\") " pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:18.932488 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:18.932439 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25442d04-e49f-454d-a510-296bc88e3deb-sys\") pod \"perf-node-gather-daemonset-tbjtv\" (UID: \"25442d04-e49f-454d-a510-296bc88e3deb\") " pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:18.932488 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:18.932478 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/25442d04-e49f-454d-a510-296bc88e3deb-proc\") pod \"perf-node-gather-daemonset-tbjtv\" (UID: \"25442d04-e49f-454d-a510-296bc88e3deb\") " pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:19.033562 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.033507 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25442d04-e49f-454d-a510-296bc88e3deb-lib-modules\") pod \"perf-node-gather-daemonset-tbjtv\" (UID: \"25442d04-e49f-454d-a510-296bc88e3deb\") " pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:19.033766 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.033575 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/25442d04-e49f-454d-a510-296bc88e3deb-podres\") pod \"perf-node-gather-daemonset-tbjtv\" (UID: \"25442d04-e49f-454d-a510-296bc88e3deb\") " pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:19.033766 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.033651 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbhvm\" (UniqueName: \"kubernetes.io/projected/25442d04-e49f-454d-a510-296bc88e3deb-kube-api-access-wbhvm\") pod \"perf-node-gather-daemonset-tbjtv\" (UID: \"25442d04-e49f-454d-a510-296bc88e3deb\") " pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:19.033766 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.033670 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25442d04-e49f-454d-a510-296bc88e3deb-sys\") pod \"perf-node-gather-daemonset-tbjtv\" (UID: \"25442d04-e49f-454d-a510-296bc88e3deb\") " pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:19.033766 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.033699 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/25442d04-e49f-454d-a510-296bc88e3deb-proc\") pod \"perf-node-gather-daemonset-tbjtv\" (UID: \"25442d04-e49f-454d-a510-296bc88e3deb\") " pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:19.033766 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.033726 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25442d04-e49f-454d-a510-296bc88e3deb-lib-modules\") pod \"perf-node-gather-daemonset-tbjtv\" (UID: \"25442d04-e49f-454d-a510-296bc88e3deb\") " pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:19.033939 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.033791 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/25442d04-e49f-454d-a510-296bc88e3deb-proc\") pod \"perf-node-gather-daemonset-tbjtv\" (UID: \"25442d04-e49f-454d-a510-296bc88e3deb\") " pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:19.033939 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.033829 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25442d04-e49f-454d-a510-296bc88e3deb-sys\") pod \"perf-node-gather-daemonset-tbjtv\" (UID: \"25442d04-e49f-454d-a510-296bc88e3deb\") " pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:19.033939 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.033894 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/25442d04-e49f-454d-a510-296bc88e3deb-podres\") pod \"perf-node-gather-daemonset-tbjtv\" (UID: \"25442d04-e49f-454d-a510-296bc88e3deb\") " pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:19.043535 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.043460 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbhvm\" (UniqueName: \"kubernetes.io/projected/25442d04-e49f-454d-a510-296bc88e3deb-kube-api-access-wbhvm\") pod \"perf-node-gather-daemonset-tbjtv\" (UID: \"25442d04-e49f-454d-a510-296bc88e3deb\") " pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:19.109165 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.109132 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:19.295338 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.295263 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv"] Jul 07 07:13:19.297702 ip-10-0-130-176 kubenswrapper[2579]: W0707 07:13:19.297664 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod25442d04_e49f_454d_a510_296bc88e3deb.slice/crio-3a3d3b125d66e9e178680bc11cf85b00b5006239d2d510ff49d03947768529bd WatchSource:0}: Error finding container 3a3d3b125d66e9e178680bc11cf85b00b5006239d2d510ff49d03947768529bd: Status 404 returned error can't find the container with id 3a3d3b125d66e9e178680bc11cf85b00b5006239d2d510ff49d03947768529bd Jul 07 07:13:19.592126 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.592011 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" event={"ID":"25442d04-e49f-454d-a510-296bc88e3deb","Type":"ContainerStarted","Data":"ce43e5bc826d21c66afdfc2b14401aece9a5517d821933616d0f07fde8941308"} Jul 07 07:13:19.592126 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.592082 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" event={"ID":"25442d04-e49f-454d-a510-296bc88e3deb","Type":"ContainerStarted","Data":"3a3d3b125d66e9e178680bc11cf85b00b5006239d2d510ff49d03947768529bd"} Jul 07 07:13:19.592472 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.592410 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:19.614528 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:19.614460 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" podStartSLOduration=1.614440003 podStartE2EDuration="1.614440003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 07:13:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 07:13:19.611278293 +0000 UTC m=+2475.451119261" watchObservedRunningTime="2026-07-07 07:13:19.614440003 +0000 UTC m=+2475.454280971" Jul 07 07:13:20.832204 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:20.832173 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sgzm5_9bf0c340-1448-46eb-aa8a-a8c1c9738ba7/dns/0.log" Jul 07 07:13:20.863592 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:20.863563 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sgzm5_9bf0c340-1448-46eb-aa8a-a8c1c9738ba7/kube-rbac-proxy/0.log" Jul 07 07:13:20.891394 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:20.891356 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-86q7n_d8deda72-2465-458f-af82-23eb7bd4156f/dns-node-resolver/0.log" Jul 07 07:13:21.561354 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:21.561310 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5658684c68-khz7t_ab14650a-fd9f-4605-ac6c-7435637be8e7/registry/0.log" Jul 07 07:13:21.668516 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:21.668484 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zjmvc_39868a40-e8ab-4965-b28a-83e1a6a46162/node-ca/0.log" Jul 07 07:13:22.639909 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:22.639874 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf42bvt5_1057d10c-fc71-480b-8f6a-1a6f4832748c/istio-proxy/0.log" Jul 07 07:13:22.770689 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:22.770650 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2gjqq_230065e6-ce67-4fbc-95bf-dff89318b04a/discovery/0.log" Jul 07 07:13:22.838507 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:22.838474 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rrfhg_b917a0bf-164b-40e9-88a6-ad5ac62e29ca/istio-proxy/0.log" Jul 07 07:13:22.872243 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:22.872208 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rrfhg_b917a0bf-164b-40e9-88a6-ad5ac62e29ca/istio-proxy/1.log" Jul 07 07:13:23.755282 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:23.755257 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vlbw7_88f4bcf7-85d5-4270-86fd-576d5c2414c8/serve-healthcheck-canary/0.log" Jul 07 07:13:24.294731 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:24.294701 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hnq4d_f5c4510a-0d58-4dcb-924d-47a74eaaa9b3/kube-rbac-proxy/0.log" Jul 07 07:13:24.313801 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:24.313775 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hnq4d_f5c4510a-0d58-4dcb-924d-47a74eaaa9b3/exporter/0.log" Jul 07 07:13:24.335847 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:24.335820 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hnq4d_f5c4510a-0d58-4dcb-924d-47a74eaaa9b3/extractor/0.log" Jul 07 07:13:25.612477 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:25.612448 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-f8h84/perf-node-gather-daemonset-tbjtv" Jul 07 07:13:26.738816 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:26.738787 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-858ccdc565-w6cgq_75b63629-9690-44c7-8263-74cba91fbbfb/maas-api/0.log" Jul 07 07:13:26.862813 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:26.862780 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77b55b5c58-wt88z_7a16f5b6-099a-4204-8c82-d28b8a48732c/manager/0.log" Jul 07 07:13:27.102453 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:27.102418 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c66c8b568-ljkmw_1cd7caa7-5d44-4786-bda1-ef41b106db29/manager/0.log" Jul 07 07:13:27.134524 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:27.134492 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c66c8b568-ljkmw_1cd7caa7-5d44-4786-bda1-ef41b106db29/copy-manifests/0.log" Jul 07 07:13:28.657635 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:28.657562 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-856bfb9fcf-8csnr_a32693bd-898c-41e1-b8da-83caea138314/manager/0.log" Jul 07 07:13:28.713114 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:28.713075 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-lpmmz_f0157843-d343-406e-ba0a-c10de7dd32cb/openshift-lws-operator/0.log" Jul 07 07:13:28.866330 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:28.866302 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf42bvt5_1057d10c-fc71-480b-8f6a-1a6f4832748c/istio-proxy/0.log" Jul 07 07:13:28.988128 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:28.988044 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2gjqq_230065e6-ce67-4fbc-95bf-dff89318b04a/discovery/0.log" Jul 07 07:13:29.030569 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:29.030538 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rrfhg_b917a0bf-164b-40e9-88a6-ad5ac62e29ca/istio-proxy/0.log" Jul 07 07:13:29.070187 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:29.070160 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rrfhg_b917a0bf-164b-40e9-88a6-ad5ac62e29ca/istio-proxy/1.log" Jul 07 07:13:30.379403 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:30.379375 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf42bvt5_1057d10c-fc71-480b-8f6a-1a6f4832748c/istio-proxy/0.log" Jul 07 07:13:30.499161 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:30.499123 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2gjqq_230065e6-ce67-4fbc-95bf-dff89318b04a/discovery/0.log" Jul 07 07:13:30.554941 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:30.554887 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rrfhg_b917a0bf-164b-40e9-88a6-ad5ac62e29ca/istio-proxy/0.log" Jul 07 07:13:30.596230 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:30.596130 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rrfhg_b917a0bf-164b-40e9-88a6-ad5ac62e29ca/istio-proxy/1.log" Jul 07 07:13:35.543647 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:35.543621 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f9mqv_acb18573-2d7c-4e7b-9429-dcad9be2ced6/kube-multus-additional-cni-plugins/0.log" Jul 07 07:13:35.569410 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:35.569379 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f9mqv_acb18573-2d7c-4e7b-9429-dcad9be2ced6/egress-router-binary-copy/0.log" Jul 07 07:13:35.599508 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:35.599480 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f9mqv_acb18573-2d7c-4e7b-9429-dcad9be2ced6/cni-plugins/0.log" Jul 07 07:13:35.622611 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:35.622581 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f9mqv_acb18573-2d7c-4e7b-9429-dcad9be2ced6/bond-cni-plugin/0.log" Jul 07 07:13:35.642531 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:35.642506 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f9mqv_acb18573-2d7c-4e7b-9429-dcad9be2ced6/routeoverride-cni/0.log" Jul 07 07:13:35.661887 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:35.661864 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f9mqv_acb18573-2d7c-4e7b-9429-dcad9be2ced6/whereabouts-cni-bincopy/0.log" Jul 07 07:13:35.679345 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:35.679321 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f9mqv_acb18573-2d7c-4e7b-9429-dcad9be2ced6/whereabouts-cni/0.log" Jul 07 07:13:35.852829 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:35.852756 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-j6ngv_228e7186-241d-4d93-9874-ee588f0a5e1d/kube-multus/0.log" Jul 07 07:13:35.976590 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:35.976549 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-thndk_e3255519-1e37-4781-8d6e-14394758a8b2/network-metrics-daemon/0.log" Jul 07 07:13:35.996103 ip-10-0-130-176 kubenswrapper[2579]: I0707 07:13:35.996072 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-thndk_e3255519-1e37-4781-8d6e-14394758a8b2/kube-rbac-proxy/0.log"